2023-02-12 14:53:31,355 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor timeout: 13 mins 2023-02-12 14:53:31,627 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be 2023-02-12 14:53:31,641 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=1, rsPorts=, rsClass=null, numDataNodes=1, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-12 14:53:31,642 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1, deleteOnExit=true 2023-02-12 14:53:31,642 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-12 14:53:31,642 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/test.cache.data in system properties and HBase conf 2023-02-12 14:53:31,643 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/hadoop.tmp.dir in system properties and HBase conf 2023-02-12 14:53:31,643 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/hadoop.log.dir in system properties and HBase conf 2023-02-12 14:53:31,644 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-12 14:53:31,644 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-12 14:53:31,644 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-12 14:53:31,755 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-12 14:53:32,172 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-12 14:53:32,176 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-12 14:53:32,176 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-12 14:53:32,176 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-12 14:53:32,176 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-12 14:53:32,177 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-12 14:53:32,177 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-12 14:53:32,177 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-12 14:53:32,178 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-12 14:53:32,178 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-12 14:53:32,179 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/nfs.dump.dir in system properties and HBase conf 2023-02-12 14:53:32,179 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/java.io.tmpdir in system properties and HBase conf 2023-02-12 14:53:32,180 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-12 14:53:32,180 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-12 14:53:32,180 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-12 14:53:32,695 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-12 14:53:32,699 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-12 14:53:32,996 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-12 14:53:33,168 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-12 14:53:33,186 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-12 14:53:33,230 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-12 14:53:33,262 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/java.io.tmpdir/Jetty_localhost_34653_hdfs____.d73b43/webapp 2023-02-12 14:53:33,433 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34653 2023-02-12 14:53:33,475 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-12 14:53:33,475 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-12 14:53:33,949 WARN [Listener at localhost/45093] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-12 14:53:34,078 WARN [Listener at localhost/45093] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-12 14:53:34,096 WARN [Listener at localhost/45093] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-12 14:53:34,104 INFO [Listener at localhost/45093] log.Slf4jLog(67): jetty-6.1.26 2023-02-12 14:53:34,108 INFO [Listener at localhost/45093] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/java.io.tmpdir/Jetty_localhost_46275_datanode____l3oad/webapp 2023-02-12 14:53:34,210 INFO [Listener at localhost/45093] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46275 2023-02-12 14:53:34,511 WARN [Listener at localhost/37945] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-12 14:53:34,927 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xae3002e764b8fc1f: Processing first storage report for DS-7dc54834-1322-4b71-9d06-301433572c1c from datanode 2330fbb2-a4ad-4699-a334-233f2dae637e 2023-02-12 14:53:34,929 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xae3002e764b8fc1f: from storage DS-7dc54834-1322-4b71-9d06-301433572c1c node DatanodeRegistration(127.0.0.1:44065, datanodeUuid=2330fbb2-a4ad-4699-a334-233f2dae637e, infoPort=44185, infoSecurePort=0, ipcPort=37945, storageInfo=lv=-57;cid=testClusterID;nsid=123215367;c=1676213612779), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-02-12 14:53:34,929 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xae3002e764b8fc1f: Processing first storage report for DS-b3393f4f-3a37-4e8c-9afd-5c158164d144 from datanode 2330fbb2-a4ad-4699-a334-233f2dae637e 2023-02-12 14:53:34,929 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xae3002e764b8fc1f: from storage DS-b3393f4f-3a37-4e8c-9afd-5c158164d144 node DatanodeRegistration(127.0.0.1:44065, datanodeUuid=2330fbb2-a4ad-4699-a334-233f2dae637e, infoPort=44185, infoSecurePort=0, ipcPort=37945, storageInfo=lv=-57;cid=testClusterID;nsid=123215367;c=1676213612779), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-12 14:53:34,999 DEBUG [Listener at localhost/37945] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be 2023-02-12 14:53:35,058 INFO [Listener at localhost/37945] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1/zookeeper_0, clientPort=61565, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-12 14:53:35,071 INFO [Listener at localhost/37945] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=61565 2023-02-12 14:53:35,080 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:35,083 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:35,722 INFO [Listener at localhost/37945] util.FSUtils(479): Created version file at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985 with version=8 2023-02-12 14:53:35,722 INFO [Listener at localhost/37945] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/hbase-staging 2023-02-12 14:53:36,022 INFO [Listener at localhost/37945] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-12 14:53:36,481 INFO [Listener at localhost/37945] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=3 2023-02-12 14:53:36,513 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 14:53:36,513 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-12 14:53:36,513 INFO [Listener at localhost/37945] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-12 14:53:36,514 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 14:53:36,514 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-12 14:53:36,650 INFO [Listener at localhost/37945] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-12 14:53:36,720 DEBUG [Listener at localhost/37945] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-12 14:53:36,812 INFO [Listener at localhost/37945] ipc.NettyRpcServer(120): Bind to /172.31.14.131:38665 2023-02-12 14:53:36,822 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:36,824 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:36,844 INFO [Listener at localhost/37945] zookeeper.RecoverableZooKeeper(93): Process identifier=master:38665 connecting to ZooKeeper ensemble=127.0.0.1:61565 2023-02-12 14:53:36,882 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:386650x0, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-12 14:53:36,885 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:38665-0x1004905e3890000 connected 2023-02-12 14:53:36,909 DEBUG [Listener at localhost/37945] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 14:53:36,910 DEBUG [Listener at localhost/37945] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 14:53:36,913 DEBUG [Listener at localhost/37945] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-12 14:53:36,920 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=38665 2023-02-12 14:53:36,921 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=38665 2023-02-12 14:53:36,921 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=38665 2023-02-12 14:53:36,921 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=38665 2023-02-12 14:53:36,922 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=38665 2023-02-12 14:53:36,929 INFO [Listener at localhost/37945] master.HMaster(439): hbase.rootdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985, hbase.cluster.distributed=false 2023-02-12 14:53:37,002 INFO [Listener at localhost/37945] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=3 2023-02-12 14:53:37,002 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 14:53:37,002 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-12 14:53:37,002 INFO [Listener at localhost/37945] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-12 14:53:37,002 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-12 14:53:37,003 INFO [Listener at localhost/37945] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-12 14:53:37,007 INFO [Listener at localhost/37945] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-12 14:53:37,010 INFO [Listener at localhost/37945] ipc.NettyRpcServer(120): Bind to /172.31.14.131:45319 2023-02-12 14:53:37,012 INFO [Listener at localhost/37945] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-12 14:53:37,018 DEBUG [Listener at localhost/37945] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-12 14:53:37,019 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:37,021 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:37,023 INFO [Listener at localhost/37945] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:45319 connecting to ZooKeeper ensemble=127.0.0.1:61565 2023-02-12 14:53:37,028 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:453190x0, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-12 14:53:37,028 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:45319-0x1004905e3890001 connected 2023-02-12 14:53:37,029 DEBUG [Listener at localhost/37945] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 14:53:37,030 DEBUG [Listener at localhost/37945] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 14:53:37,031 DEBUG [Listener at localhost/37945] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-12 14:53:37,031 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=45319 2023-02-12 14:53:37,032 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=45319 2023-02-12 14:53:37,032 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=45319 2023-02-12 14:53:37,032 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=45319 2023-02-12 14:53:37,033 DEBUG [Listener at localhost/37945] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=45319 2023-02-12 14:53:37,036 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:37,044 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-12 14:53:37,046 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:37,062 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-12 14:53:37,062 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-12 14:53:37,063 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:37,063 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-12 14:53:37,065 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,38665,1676213615861 from backup master directory 2023-02-12 14:53:37,065 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-12 14:53:37,067 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:37,067 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-12 14:53:37,068 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-12 14:53:37,068 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:37,071 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-12 14:53:37,072 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-12 14:53:37,151 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/hbase.id with ID: d5460da4-ebb4-4818-b0cf-7f668f6d7a0e 2023-02-12 14:53:37,190 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:37,205 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:37,243 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x0b3d6d3c to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:53:37,274 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2bb61f1b, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:53:37,298 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-12 14:53:37,300 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-12 14:53:37,320 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-12 14:53:37,320 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-12 14:53:37,323 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-12 14:53:37,327 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-12 14:53:37,328 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 14:53:37,360 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store-tmp 2023-02-12 14:53:37,385 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:37,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-12 14:53:37,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:53:37,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:53:37,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-12 14:53:37,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:53:37,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:53:37,386 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676213617385Disabling compacts and flushes for region at 1676213617385Disabling writes for close at 1676213617386 (+1 ms)Writing region close event to WAL at 1676213617386Closed at 1676213617386 2023-02-12 14:53:37,388 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/WALs/jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:37,409 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C38665%2C1676213615861, suffix=, logDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/WALs/jenkins-hbase4.apache.org,38665,1676213615861, archiveDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/oldWALs, maxLogs=10 2023-02-12 14:53:37,462 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44065,DS-7dc54834-1322-4b71-9d06-301433572c1c,DISK] 2023-02-12 14:53:37,470 DEBUG [RS-EventLoopGroup-3-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-12 14:53:37,526 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/WALs/jenkins-hbase4.apache.org,38665,1676213615861/jenkins-hbase4.apache.org%2C38665%2C1676213615861.1676213617419 2023-02-12 14:53:37,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44065,DS-7dc54834-1322-4b71-9d06-301433572c1c,DISK]] 2023-02-12 14:53:37,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:53:37,527 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:37,530 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-12 14:53:37,531 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-12 14:53:37,584 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-12 14:53:37,590 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-12 14:53:37,613 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-12 14:53:37,626 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:37,632 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-12 14:53:37,633 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-12 14:53:37,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-12 14:53:37,653 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:53:37,655 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60534296, jitterRate=-0.0979686975479126}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:53:37,655 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676213617555Initializing all the Stores at 1676213617557 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676213617558 (+1 ms)Cleaning up temporary data from old regions at 1676213617639 (+81 ms)Cleaning up detritus from prior splits at 1676213617640 (+1 ms)Region opened successfully at 1676213617655 (+15 ms) 2023-02-12 14:53:37,656 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-12 14:53:37,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=5 2023-02-12 14:53:37,675 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 5 core workers (bigger of cpus/4 or 16) with max (burst) worker count=50 2023-02-12 14:53:37,677 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-12 14:53:37,679 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-12 14:53:37,709 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 30 msec 2023-02-12 14:53:37,710 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-12 14:53:37,734 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-12 14:53:37,739 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-12 14:53:37,763 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-12 14:53:37,767 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-12 14:53:37,769 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-12 14:53:37,774 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-12 14:53:37,778 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-12 14:53:37,780 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:37,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-12 14:53:37,782 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-12 14:53:37,798 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-12 14:53:37,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-12 14:53:37,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-12 14:53:37,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:37,804 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase4.apache.org,38665,1676213615861, sessionid=0x1004905e3890000, setting cluster-up flag (Was=false) 2023-02-12 14:53:37,819 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:37,824 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-12 14:53:37,825 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:37,831 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:37,836 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-12 14:53:37,838 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:37,841 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp 2023-02-12 14:53:37,938 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(952): ClusterId : d5460da4-ebb4-4818-b0cf-7f668f6d7a0e 2023-02-12 14:53:37,943 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-12 14:53:37,948 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-12 14:53:37,949 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-12 14:53:37,950 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-12 14:53:37,953 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-12 14:53:37,954 DEBUG [RS:0;jenkins-hbase4:45319] zookeeper.ReadOnlyZKClient(139): Connect 0x46c2a243 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:53:37,958 DEBUG [RS:0;jenkins-hbase4:45319] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@480554eb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:53:37,958 DEBUG [RS:0;jenkins-hbase4:45319] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4c3ac933, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-12 14:53:37,960 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:37,962 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676213647962 2023-02-12 14:53:37,964 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-12 14:53:37,969 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-12 14:53:37,969 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-12 14:53:37,974 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-12 14:53:37,977 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-12 14:53:37,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-12 14:53:37,983 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-12 14:53:37,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-12 14:53:37,984 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-12 14:53:37,985 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:37,986 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-12 14:53:37,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-12 14:53:37,987 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-12 14:53:37,989 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:45319 2023-02-12 14:53:37,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-12 14:53:37,990 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-12 14:53:37,992 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676213617992,5,FailOnTimeoutGroup] 2023-02-12 14:53:37,992 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676213617992,5,FailOnTimeoutGroup] 2023-02-12 14:53:37,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:37,992 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-12 14:53:37,993 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:37,994 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:37,994 INFO [RS:0;jenkins-hbase4:45319] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-12 14:53:37,995 INFO [RS:0;jenkins-hbase4:45319] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-12 14:53:37,995 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1023): About to register with Master. 2023-02-12 14:53:37,998 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase4.apache.org,38665,1676213615861 with isa=jenkins-hbase4.apache.org/172.31.14.131:45319, startcode=1676213617001 2023-02-12 14:53:38,016 DEBUG [RS:0;jenkins-hbase4:45319] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-12 14:53:38,062 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35295, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-12 14:53:38,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:38,087 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985 2023-02-12 14:53:38,087 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost:45093 2023-02-12 14:53:38,087 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-12 14:53:38,093 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 14:53:38,093 DEBUG [RS:0;jenkins-hbase4:45319] zookeeper.ZKUtil(162): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:38,094 WARN [RS:0;jenkins-hbase4:45319] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-12 14:53:38,094 INFO [RS:0;jenkins-hbase4:45319] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 14:53:38,095 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1947): logDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/WALs/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:38,096 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,45319,1676213617001] 2023-02-12 14:53:38,103 DEBUG [RS:0;jenkins-hbase4:45319] zookeeper.ZKUtil(162): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:38,113 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-12 14:53:38,120 INFO [RS:0;jenkins-hbase4:45319] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-12 14:53:38,138 INFO [RS:0;jenkins-hbase4:45319] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-12 14:53:38,142 INFO [RS:0;jenkins-hbase4:45319] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:53:38,142 INFO [RS:0;jenkins-hbase4:45319] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:38,143 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-12 14:53:38,149 INFO [RS:0;jenkins-hbase4:45319] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=100, unit=MILLISECONDS is enabled. 2023-02-12 14:53:38,149 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,150 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,150 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,150 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,150 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,150 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-02-12 14:53:38,150 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,150 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,151 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,151 DEBUG [RS:0;jenkins-hbase4:45319] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-02-12 14:53:38,151 INFO [RS:0;jenkins-hbase4:45319] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:38,152 INFO [RS:0;jenkins-hbase4:45319] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:38,152 INFO [RS:0;jenkins-hbase4:45319] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:38,166 INFO [RS:0;jenkins-hbase4:45319] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-12 14:53:38,168 INFO [RS:0;jenkins-hbase4:45319] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,45319,1676213617001-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:38,183 INFO [RS:0;jenkins-hbase4:45319] regionserver.Replication(203): jenkins-hbase4.apache.org,45319,1676213617001 started 2023-02-12 14:53:38,183 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1638): Serving as jenkins-hbase4.apache.org,45319,1676213617001, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:45319, sessionid=0x1004905e3890001 2023-02-12 14:53:38,183 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-12 14:53:38,183 DEBUG [RS:0;jenkins-hbase4:45319] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:38,183 DEBUG [RS:0;jenkins-hbase4:45319] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45319,1676213617001' 2023-02-12 14:53:38,183 DEBUG [RS:0;jenkins-hbase4:45319] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:38,184 DEBUG [RS:0;jenkins-hbase4:45319] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:38,185 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-12 14:53:38,185 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-12 14:53:38,185 DEBUG [RS:0;jenkins-hbase4:45319] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:38,185 DEBUG [RS:0;jenkins-hbase4:45319] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,45319,1676213617001' 2023-02-12 14:53:38,185 DEBUG [RS:0;jenkins-hbase4:45319] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-12 14:53:38,185 DEBUG [RS:0;jenkins-hbase4:45319] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-12 14:53:38,186 DEBUG [RS:0;jenkins-hbase4:45319] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-12 14:53:38,186 INFO [RS:0;jenkins-hbase4:45319] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-12 14:53:38,186 INFO [RS:0;jenkins-hbase4:45319] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-12 14:53:38,296 INFO [RS:0;jenkins-hbase4:45319] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45319%2C1676213617001, suffix=, logDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/WALs/jenkins-hbase4.apache.org,45319,1676213617001, archiveDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/oldWALs, maxLogs=32 2023-02-12 14:53:38,314 DEBUG [RS-EventLoopGroup-3-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44065,DS-7dc54834-1322-4b71-9d06-301433572c1c,DISK] 2023-02-12 14:53:38,317 INFO [RS:0;jenkins-hbase4:45319] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/WALs/jenkins-hbase4.apache.org,45319,1676213617001/jenkins-hbase4.apache.org%2C45319%2C1676213617001.1676213618298 2023-02-12 14:53:38,317 DEBUG [RS:0;jenkins-hbase4:45319] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44065,DS-7dc54834-1322-4b71-9d06-301433572c1c,DISK]] 2023-02-12 14:53:38,408 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-12 14:53:38,409 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-12 14:53:38,409 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985 2023-02-12 14:53:38,822 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:38,825 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-12 14:53:38,827 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/info 2023-02-12 14:53:38,828 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-12 14:53:38,829 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:38,830 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-12 14:53:38,832 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/rep_barrier 2023-02-12 14:53:38,833 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-12 14:53:38,834 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:38,834 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-12 14:53:38,836 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/table 2023-02-12 14:53:38,837 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-12 14:53:38,837 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:38,839 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740 2023-02-12 14:53:38,839 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740 2023-02-12 14:53:38,843 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-12 14:53:38,844 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-12 14:53:38,848 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:53:38,849 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61951562, jitterRate=-0.0768497884273529}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-12 14:53:38,849 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676213618823Initializing all the Stores at 1676213618824 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676213618824Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676213618824Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676213618824Cleaning up temporary data from old regions at 1676213618841 (+17 ms)Cleaning up detritus from prior splits at 1676213618842 (+1 ms)Region opened successfully at 1676213618849 (+7 ms) 2023-02-12 14:53:38,850 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-12 14:53:38,850 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-12 14:53:38,850 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-12 14:53:38,850 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-12 14:53:38,850 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-12 14:53:38,851 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-12 14:53:38,851 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676213618850Disabling compacts and flushes for region at 1676213618850Disabling writes for close at 1676213618850Writing region close event to WAL at 1676213618851 (+1 ms)Closed at 1676213618851 2023-02-12 14:53:38,856 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-12 14:53:38,856 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-12 14:53:38,864 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-12 14:53:38,874 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-12 14:53:38,877 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-12 14:53:39,029 DEBUG [jenkins-hbase4:38665] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=1, allServersCount=1 2023-02-12 14:53:39,032 INFO [PEWorker-3] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45319,1676213617001, state=OPENING 2023-02-12 14:53:39,039 DEBUG [PEWorker-3] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-12 14:53:39,040 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:39,041 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-12 14:53:39,043 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:53:39,226 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:39,228 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-12 14:53:39,231 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36496, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-12 14:53:39,243 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-12 14:53:39,243 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-12 14:53:39,246 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C45319%2C1676213617001.meta, suffix=.meta, logDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/WALs/jenkins-hbase4.apache.org,45319,1676213617001, archiveDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/oldWALs, maxLogs=32 2023-02-12 14:53:39,264 DEBUG [RS-EventLoopGroup-3-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44065,DS-7dc54834-1322-4b71-9d06-301433572c1c,DISK] 2023-02-12 14:53:39,269 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/WALs/jenkins-hbase4.apache.org,45319,1676213617001/jenkins-hbase4.apache.org%2C45319%2C1676213617001.meta.1676213619248.meta 2023-02-12 14:53:39,269 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44065,DS-7dc54834-1322-4b71-9d06-301433572c1c,DISK]] 2023-02-12 14:53:39,269 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:53:39,271 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-12 14:53:39,286 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-12 14:53:39,291 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-12 14:53:39,297 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-12 14:53:39,297 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:39,297 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-12 14:53:39,297 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-12 14:53:39,300 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-12 14:53:39,301 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/info 2023-02-12 14:53:39,302 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/info 2023-02-12 14:53:39,302 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-12 14:53:39,303 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:39,303 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-12 14:53:39,305 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/rep_barrier 2023-02-12 14:53:39,305 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/rep_barrier 2023-02-12 14:53:39,305 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-12 14:53:39,306 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:39,307 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-12 14:53:39,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/table 2023-02-12 14:53:39,308 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/table 2023-02-12 14:53:39,309 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-12 14:53:39,309 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:39,311 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740 2023-02-12 14:53:39,313 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740 2023-02-12 14:53:39,317 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-12 14:53:39,319 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-12 14:53:39,321 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74982247, jitterRate=0.11732254922389984}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-12 14:53:39,321 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676213619297Writing region info on filesystem at 1676213619298 (+1 ms)Initializing all the Stores at 1676213619299 (+1 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676213619299Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676213619299Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676213619300 (+1 ms)Cleaning up temporary data from old regions at 1676213619315 (+15 ms)Cleaning up detritus from prior splits at 1676213619316 (+1 ms)Running coprocessor post-open hooks at 1676213619321 (+5 ms)Region opened successfully at 1676213619321 2023-02-12 14:53:39,330 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676213619218 2023-02-12 14:53:39,347 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-12 14:53:39,348 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-12 14:53:39,348 INFO [PEWorker-5] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,45319,1676213617001, state=OPEN 2023-02-12 14:53:39,351 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-12 14:53:39,351 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-12 14:53:39,355 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-12 14:53:39,355 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,45319,1676213617001 in 308 msec 2023-02-12 14:53:39,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-12 14:53:39,361 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 493 msec 2023-02-12 14:53:39,366 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 1.4820 sec 2023-02-12 14:53:39,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676213617059, completionTime=-1 2023-02-12 14:53:39,366 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=1; waited=0ms, expected min=1 server(s), max=1 server(s), master is running 2023-02-12 14:53:39,366 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-12 14:53:39,425 DEBUG [hconnection-0x6b94224d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:53:39,428 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36500, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:53:39,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=1 2023-02-12 14:53:39,444 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676213679444 2023-02-12 14:53:39,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676213739445 2023-02-12 14:53:39,445 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 78 msec 2023-02-12 14:53:39,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38665,1676213615861-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:39,468 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38665,1676213615861-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:39,469 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38665,1676213615861-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:39,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:38665, period=300000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:39,471 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-12 14:53:39,477 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-12 14:53:39,486 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-12 14:53:39,487 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-12 14:53:39,496 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-12 14:53:39,498 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:53:39,501 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:53:39,520 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:39,522 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664 empty. 2023-02-12 14:53:39,523 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:39,523 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-12 14:53:39,543 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-12 14:53:39,546 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => b4d026fc325cd5a11dc0dcb67fe52664, NAME => 'hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:53:39,958 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:39,958 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing b4d026fc325cd5a11dc0dcb67fe52664, disabling compactions & flushes 2023-02-12 14:53:39,958 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:53:39,958 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:53:39,958 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. after waiting 0 ms 2023-02-12 14:53:39,958 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:53:39,958 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:53:39,958 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for b4d026fc325cd5a11dc0dcb67fe52664: Waiting for close lock at 1676213619958Disabling compacts and flushes for region at 1676213619958Disabling writes for close at 1676213619958Writing region close event to WAL at 1676213619958Closed at 1676213619958 2023-02-12 14:53:39,963 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:53:39,978 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676213619966"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213619966"}]},"ts":"1676213619966"} 2023-02-12 14:53:40,005 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:53:40,007 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:53:40,011 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213620007"}]},"ts":"1676213620007"} 2023-02-12 14:53:40,016 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-12 14:53:40,025 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b4d026fc325cd5a11dc0dcb67fe52664, ASSIGN}] 2023-02-12 14:53:40,028 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b4d026fc325cd5a11dc0dcb67fe52664, ASSIGN 2023-02-12 14:53:40,030 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=b4d026fc325cd5a11dc0dcb67fe52664, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:53:40,181 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b4d026fc325cd5a11dc0dcb67fe52664, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:40,181 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676213620181"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213620181"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213620181"}]},"ts":"1676213620181"} 2023-02-12 14:53:40,186 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure b4d026fc325cd5a11dc0dcb67fe52664, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:53:40,345 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:53:40,346 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => b4d026fc325cd5a11dc0dcb67fe52664, NAME => 'hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:53:40,347 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:40,347 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:40,347 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:40,347 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:40,349 INFO [StoreOpener-b4d026fc325cd5a11dc0dcb67fe52664-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:40,351 DEBUG [StoreOpener-b4d026fc325cd5a11dc0dcb67fe52664-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/info 2023-02-12 14:53:40,351 DEBUG [StoreOpener-b4d026fc325cd5a11dc0dcb67fe52664-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/info 2023-02-12 14:53:40,352 INFO [StoreOpener-b4d026fc325cd5a11dc0dcb67fe52664-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b4d026fc325cd5a11dc0dcb67fe52664 columnFamilyName info 2023-02-12 14:53:40,353 INFO [StoreOpener-b4d026fc325cd5a11dc0dcb67fe52664-1] regionserver.HStore(310): Store=b4d026fc325cd5a11dc0dcb67fe52664/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:40,354 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:40,355 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:40,359 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:53:40,363 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:53:40,364 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened b4d026fc325cd5a11dc0dcb67fe52664; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64178469, jitterRate=-0.0436662882566452}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:53:40,364 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for b4d026fc325cd5a11dc0dcb67fe52664: Running coprocessor pre-open hook at 1676213620347Writing region info on filesystem at 1676213620347Initializing all the Stores at 1676213620349 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676213620349Cleaning up temporary data from old regions at 1676213620356 (+7 ms)Cleaning up detritus from prior splits at 1676213620356Running coprocessor post-open hooks at 1676213620364 (+8 ms)Region opened successfully at 1676213620364 2023-02-12 14:53:40,365 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664., pid=6, masterSystemTime=1676213620339 2023-02-12 14:53:40,369 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:53:40,369 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:53:40,370 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b4d026fc325cd5a11dc0dcb67fe52664, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:40,371 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676213620370"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213620370"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213620370"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213620370"}]},"ts":"1676213620370"} 2023-02-12 14:53:40,378 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-12 14:53:40,378 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure b4d026fc325cd5a11dc0dcb67fe52664, server=jenkins-hbase4.apache.org,45319,1676213617001 in 189 msec 2023-02-12 14:53:40,381 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-12 14:53:40,381 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=b4d026fc325cd5a11dc0dcb67fe52664, ASSIGN in 353 msec 2023-02-12 14:53:40,383 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:53:40,383 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213620383"}]},"ts":"1676213620383"} 2023-02-12 14:53:40,385 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-12 14:53:40,389 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:53:40,392 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 901 msec 2023-02-12 14:53:40,399 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-12 14:53:40,400 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-12 14:53:40,400 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:53:40,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-12 14:53:40,458 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-12 14:53:40,465 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 36 msec 2023-02-12 14:53:40,474 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-12 14:53:40,483 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-12 14:53:40,488 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 13 msec 2023-02-12 14:53:40,499 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-12 14:53:40,501 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-12 14:53:40,503 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 3.435sec 2023-02-12 14:53:40,505 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-12 14:53:40,506 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-12 14:53:40,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-12 14:53:40,507 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38665,1676213615861-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-12 14:53:40,509 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,38665,1676213615861-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-12 14:53:40,519 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-12 14:53:40,544 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(139): Connect 0x13c53277 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:53:40,549 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a675bd6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:53:40,561 DEBUG [hconnection-0x29ab39ac-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:53:40,572 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36510, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:53:40,583 INFO [Listener at localhost/37945] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:53:40,584 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(139): Connect 0x5831e476 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:53:40,589 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6f46f74d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:53:40,589 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:53:40,611 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=212, OpenFileDescriptor=475, MaxFileDescriptor=60000, SystemLoadAverage=294, ProcessCount=168, AvailableMemoryMB=4867 2023-02-12 14:53:40,621 DEBUG [Listener at localhost/37945] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:53:40,624 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:37206, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:53:40,636 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-12 14:53:40,638 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:40,640 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:53:40,642 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:53:40,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthly" procId is: 9 2023-02-12 14:53:40,644 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:40,646 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7 empty. 2023-02-12 14:53:40,647 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:40,648 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthly regions 2023-02-12 14:53:40,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-12 14:53:40,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-12 14:53:40,960 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-12 14:53:41,065 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-12 14:53:41,067 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0384a0af3ce8daed2b5dc2a6da84edf7, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:53:41,079 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:41,080 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1603): Closing 0384a0af3ce8daed2b5dc2a6da84edf7, disabling compactions & flushes 2023-02-12 14:53:41,080 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,080 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,080 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. after waiting 0 ms 2023-02-12 14:53:41,080 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,080 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,080 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthly-pool-0] regionserver.HRegion(1557): Region close journal for 0384a0af3ce8daed2b5dc2a6da84edf7: Waiting for close lock at 1676213621080Disabling compacts and flushes for region at 1676213621080Disabling writes for close at 1676213621080Writing region close event to WAL at 1676213621080Closed at 1676213621080 2023-02-12 14:53:41,084 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:53:41,085 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676213621085"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213621085"}]},"ts":"1676213621085"} 2023-02-12 14:53:41,088 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:53:41,089 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:53:41,089 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213621089"}]},"ts":"1676213621089"} 2023-02-12 14:53:41,091 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLING in hbase:meta 2023-02-12 14:53:41,095 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, ASSIGN}] 2023-02-12 14:53:41,096 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, ASSIGN 2023-02-12 14:53:41,098 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:53:41,249 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=0384a0af3ce8daed2b5dc2a6da84edf7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:41,249 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676213621249"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213621249"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213621249"}]},"ts":"1676213621249"} 2023-02-12 14:53:41,252 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 0384a0af3ce8daed2b5dc2a6da84edf7, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:53:41,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-12 14:53:41,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0384a0af3ce8daed2b5dc2a6da84edf7, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:53:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:41,415 INFO [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:41,417 DEBUG [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:41,418 DEBUG [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:41,420 INFO [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0384a0af3ce8daed2b5dc2a6da84edf7 columnFamilyName family1 2023-02-12 14:53:41,423 INFO [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] regionserver.HStore(310): Store=0384a0af3ce8daed2b5dc2a6da84edf7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:41,425 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:41,426 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:41,431 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:41,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:53:41,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0384a0af3ce8daed2b5dc2a6da84edf7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63074368, jitterRate=-0.060118675231933594}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:53:41,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0384a0af3ce8daed2b5dc2a6da84edf7: Running coprocessor pre-open hook at 1676213621412Writing region info on filesystem at 1676213621412Initializing all the Stores at 1676213621414 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676213621414Cleaning up temporary data from old regions at 1676213621427 (+13 ms)Cleaning up detritus from prior splits at 1676213621428 (+1 ms)Running coprocessor post-open hooks at 1676213621436 (+8 ms)Region opened successfully at 1676213621436 2023-02-12 14:53:41,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7., pid=11, masterSystemTime=1676213621405 2023-02-12 14:53:41,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,441 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=0384a0af3ce8daed2b5dc2a6da84edf7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:41,442 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676213621441"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213621441"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213621441"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213621441"}]},"ts":"1676213621441"} 2023-02-12 14:53:41,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-12 14:53:41,448 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 0384a0af3ce8daed2b5dc2a6da84edf7, server=jenkins-hbase4.apache.org,45319,1676213617001 in 193 msec 2023-02-12 14:53:41,451 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-12 14:53:41,452 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, ASSIGN in 354 msec 2023-02-12 14:53:41,453 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:53:41,453 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213621453"}]},"ts":"1676213621453"} 2023-02-12 14:53:41,456 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthly, state=ENABLED in hbase:meta 2023-02-12 14:53:41,459 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:53:41,461 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 823 msec 2023-02-12 14:53:41,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=9 2023-02-12 14:53:41,764 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthly, procId: 9 completed 2023-02-12 14:53:41,786 DEBUG [hconnection-0x130f47b7-metaLookup-shared--pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:53:41,789 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36518, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:53:41,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:41,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,976 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:41,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:41,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:41,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:41,978 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,978 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,980 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:41,980 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:41,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:41,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:41,981 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:41,981 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:41,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,981 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:41,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,982 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:41,984 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:41,984 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:41,985 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:41,985 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:41,985 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:41,986 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:41,986 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:41,987 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-12 14:53:42,532 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a083ed1d440a416185d5fee3670f8b80 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a083ed1d440a416185d5fee3670f8b80 2023-02-12 14:53:42,548 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/c9b044972b7a40ffac92814d3a926e39 2023-02-12 14:53:42,558 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/c9b044972b7a40ffac92814d3a926e39 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c9b044972b7a40ffac92814d3a926e39 2023-02-12 14:53:42,567 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c9b044972b7a40ffac92814d3a926e39, entries=1000, sequenceid=5, filesize=175.8 K 2023-02-12 14:53:42,570 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 582ms, sequenceid=5, compaction requested=false 2023-02-12 14:53:42,571 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:42,573 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:42,574 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:42,574 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:42,575 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:42,575 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,575 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:42,575 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:42,577 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,577 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:42,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:42,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:42,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:42,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:42,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:42,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:42,580 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:42,580 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7593812e[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:42,580 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,582 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,582 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:42,582 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:42,582 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:42,582 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,582 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:42,584 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,584 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:42,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:42,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:42,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:42,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:42,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:42,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:42,590 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:42,590 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:42,590 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:42,590 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:42,590 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,590 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:42,593 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,593 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:42,593 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,593 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:42,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:42,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:42,594 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:42,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,596 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:42,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,603 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,603 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:42,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:42,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:42,604 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:42,604 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:42,604 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:42,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:42,604 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,604 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,604 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,604 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,604 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:42,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:42,607 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:42,607 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:42,707 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:42,713 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:42,724 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:42,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:42,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,727 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:42,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:42,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:42,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:42,728 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,728 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,730 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:42,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:42,730 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,730 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:42,730 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:42,731 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:42,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,731 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:42,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,732 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,732 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:42,732 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:42,732 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:42,732 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:42,733 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:42,733 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:42,733 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:42,733 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:42,733 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:42,733 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-12 14:53:42,763 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b20355a1dcf549f0b281418084bd9637 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128b20355a1dcf549f0b281418084bd9637 2023-02-12 14:53:43,177 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/098010b74de345f88e97251e8eca3d3f 2023-02-12 14:53:43,187 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/098010b74de345f88e97251e8eca3d3f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/098010b74de345f88e97251e8eca3d3f 2023-02-12 14:53:43,195 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/098010b74de345f88e97251e8eca3d3f, entries=1, sequenceid=9, filesize=5.0 K 2023-02-12 14:53:43,196 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 463ms, sequenceid=9, compaction requested=false 2023-02-12 14:53:43,196 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:43,196 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:43,196 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:43,197 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:43,197 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,197 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:43,197 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:43,199 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,199 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:43,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:43,200 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,200 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:43,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:43,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:43,202 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@588a11e1[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:43,202 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:43,202 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,203 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,203 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:43,204 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:43,204 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,204 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:43,206 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:43,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:43,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:43,206 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:43,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:43,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:43,209 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:43,209 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:43,209 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:43,209 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:43,209 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,209 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:43,216 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,216 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:43,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,216 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:43,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:43,217 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,217 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:43,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,219 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:43,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,223 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:43,223 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:43,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:43,223 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:43,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:43,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:43,223 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:43,223 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,223 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-12 14:53:43,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:43,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:43,224 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:43,225 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:43,325 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:43,327 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:43,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:43,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:43,338 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,338 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:43,338 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:43,339 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:43,339 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:43,339 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,340 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,341 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:43,341 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:43,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:43,342 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,342 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:43,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,342 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:43,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,343 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,343 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:43,343 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,343 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:43,343 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:43,344 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:43,344 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:43,344 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:43,344 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:43,344 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:43,344 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:43,772 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201777f847aa08a465fa599e25efd56f6c1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201777f847aa08a465fa599e25efd56f6c1 2023-02-12 14:53:43,785 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/a050537be3044f08b06c3c125b411995 2023-02-12 14:53:43,793 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/a050537be3044f08b06c3c125b411995 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/a050537be3044f08b06c3c125b411995 2023-02-12 14:53:43,801 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/a050537be3044f08b06c3c125b411995, entries=1, sequenceid=13, filesize=5.0 K 2023-02-12 14:53:43,802 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 458ms, sequenceid=13, compaction requested=true 2023-02-12 14:53:43,802 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:43,802 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:43,802 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:43,803 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:43,803 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,803 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:43,803 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:43,805 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,805 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:43,806 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,806 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@239a0b94[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:43,808 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:43,808 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,810 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:43,810 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,810 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:43,812 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:43,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:43,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:43,815 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:43,815 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:43,815 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:43,815 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,815 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:43,817 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,817 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,818 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:43,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:43,818 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:43,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,821 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,825 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,825 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:43,825 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:43,825 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:43,825 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-12 14:53:43,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:43,826 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:43,826 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:43,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:43,827 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:43,827 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:43,927 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:43,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:43,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:43,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:43,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,941 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:43,941 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:43,941 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:43,941 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:43,942 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,942 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,945 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,945 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:43,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:43,945 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:43,945 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:43,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:43,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,947 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:43,947 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:43,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:43,947 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:43,947 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:43,948 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:44,117 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-12 14:53:44,118 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-12 14:53:44,380 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205fc9befbeb5dd4c8d8ed2030efe6a9e3b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205fc9befbeb5dd4c8d8ed2030efe6a9e3b 2023-02-12 14:53:44,794 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/89b41e5ba0a5447f971263bba825b81a 2023-02-12 14:53:44,806 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/89b41e5ba0a5447f971263bba825b81a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/89b41e5ba0a5447f971263bba825b81a 2023-02-12 14:53:44,817 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/89b41e5ba0a5447f971263bba825b81a, entries=1, sequenceid=17, filesize=5.0 K 2023-02-12 14:53:44,819 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 871ms, sequenceid=17, compaction requested=true 2023-02-12 14:53:44,819 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:44,819 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:44,819 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:44,820 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:44,820 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,820 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:53:44,820 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:44,820 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:44,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:44,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:44,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:44,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:44,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,829 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:44,830 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,830 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:44,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:44,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7702a1de[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:44,830 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:44,830 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,832 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:44,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:44,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:44,832 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,832 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:44,834 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:44,834 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:44,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:44,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:44,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:44,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:44,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:44,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:44,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:44,840 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:44,840 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:44,840 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:44,840 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,840 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:44,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,844 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:44,844 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:44,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:44,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:44,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:44,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,853 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,853 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,853 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:44,854 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:44,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:44,854 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:44,854 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:44,853 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:44,855 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:44,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:44,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:44,854 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:44,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:44,955 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:44,957 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:44,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:44,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:44,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,968 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:44,968 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:44,969 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:44,969 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:44,970 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,970 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,971 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,971 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:44,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:44,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:44,971 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:44,971 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:44,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,973 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:44,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,973 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:44,974 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:44,974 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:44,974 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:44,974 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:44,974 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:44,974 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:44,974 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:44,975 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:44,975 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:45,401 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228ac81b98035454d2292278e6a81f1ba0b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228ac81b98035454d2292278e6a81f1ba0b 2023-02-12 14:53:45,813 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c 2023-02-12 14:53:45,822 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c 2023-02-12 14:53:45,831 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c, entries=1, sequenceid=21, filesize=5.0 K 2023-02-12 14:53:45,832 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 857ms, sequenceid=21, compaction requested=true 2023-02-12 14:53:45,833 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:45,833 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:45,833 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:45,839 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,839 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:45,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:45,840 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,840 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:45,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:45,842 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1e71d4e1[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:45,842 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:45,842 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:45,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:45,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:45,843 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,843 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:45,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:45,845 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:45,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:45,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:45,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:45,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:45,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:45,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:45,848 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:45,848 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:45,848 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:45,848 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:45,848 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,848 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:45,850 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,850 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,850 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:45,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:45,850 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:45,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:45,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,852 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:45,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:45,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:45,856 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-12 14:53:45,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:45,856 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:45,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:45,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:45,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:45,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,857 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:45,857 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:45,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:45,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:45,957 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:45,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:45,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:45,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:45,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,968 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:45,968 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:45,969 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:45,969 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:45,970 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,970 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,971 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:45,971 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:45,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:45,972 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:45,972 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:45,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,972 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:45,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,972 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,973 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:45,973 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:45,973 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:45,973 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:45,974 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:45,974 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:46,016 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-12 14:53:46,017 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-12 14:53:46,396 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512317f5436f7baa44d648483de22a3efd109 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317f5436f7baa44d648483de22a3efd109 2023-02-12 14:53:46,807 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/22544ede5bc24332a770cc327a76d202 2023-02-12 14:53:46,817 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/22544ede5bc24332a770cc327a76d202 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/22544ede5bc24332a770cc327a76d202 2023-02-12 14:53:46,830 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/22544ede5bc24332a770cc327a76d202, entries=1, sequenceid=25, filesize=5.0 K 2023-02-12 14:53:46,831 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 857ms, sequenceid=25, compaction requested=true 2023-02-12 14:53:46,831 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:46,831 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:46,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:46,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:46,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:46,832 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:46,834 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,834 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:46,834 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,834 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:46,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:46,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:46,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:46,836 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@167ccbd7[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:46,836 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:46,836 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,837 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,837 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:46,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:46,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:46,838 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,838 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:46,840 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:46,840 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,840 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:46,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:46,840 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:46,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:46,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:46,844 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:46,844 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:46,844 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:46,844 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,844 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:46,846 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,846 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:46,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,846 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:46,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:46,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:46,846 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:46,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,848 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:46,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:46,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:46,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:46,851 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:46,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:46,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:46,851 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:46,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,852 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,852 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,852 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:46,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:46,853 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:46,853 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:46,953 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:46,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:46,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:46,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:46,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,964 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:46,964 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:46,964 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:46,965 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:46,965 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,965 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,966 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,966 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:46,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:46,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:46,967 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:46,967 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:46,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,967 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:46,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,968 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,968 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:46,968 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:46,968 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:46,969 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:46,969 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:46,969 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:46,969 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:46,969 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:46,969 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:46,969 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:47,390 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101d502652d90e14739b2c3230d96688ab1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101d502652d90e14739b2c3230d96688ab1 2023-02-12 14:53:47,799 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/2428093bad3b4a88b8641be164950798 2023-02-12 14:53:47,807 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/2428093bad3b4a88b8641be164950798 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/2428093bad3b4a88b8641be164950798 2023-02-12 14:53:47,816 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/2428093bad3b4a88b8641be164950798, entries=1, sequenceid=29, filesize=5.0 K 2023-02-12 14:53:47,817 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 848ms, sequenceid=29, compaction requested=true 2023-02-12 14:53:47,817 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:47,817 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:47,817 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:47,818 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:47,818 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,818 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:47,818 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:47,825 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,825 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:47,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:47,825 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,826 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:47,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:47,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:47,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:47,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:47,827 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6549a427[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:47,827 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:47,827 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,828 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,828 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:47,829 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:47,829 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:47,829 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,829 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:47,830 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:47,830 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:47,830 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,830 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:47,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:47,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:47,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:47,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:47,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:47,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:47,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:47,833 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:47,833 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:47,833 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:47,833 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,833 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:47,835 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,835 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:47,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:47,835 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:47,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:47,835 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:47,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:47,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:47,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,837 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:47,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:47,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:47,843 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:47,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:47,844 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:47,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:47,844 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,844 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-12 14:53:47,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:47,844 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,844 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,844 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,844 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:47,844 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:47,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:47,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:47,944 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:47,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:47,952 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:47,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:47,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,955 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:47,955 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:47,955 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:47,955 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:47,956 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,956 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,957 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,957 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:47,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:47,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:47,957 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:47,957 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:47,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,958 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:47,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,958 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,959 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:47,959 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:47,959 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:47,959 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:47,960 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:47,960 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:48,381 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601030b0b60a028ba446d8be95e0bc96115f4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601030b0b60a028ba446d8be95e0bc96115f4 2023-02-12 14:53:48,790 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/3ecd02539aa3467da683f9107cb6f0a1 2023-02-12 14:53:48,799 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/3ecd02539aa3467da683f9107cb6f0a1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/3ecd02539aa3467da683f9107cb6f0a1 2023-02-12 14:53:48,806 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/3ecd02539aa3467da683f9107cb6f0a1, entries=1, sequenceid=33, filesize=5.0 K 2023-02-12 14:53:48,807 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 847ms, sequenceid=33, compaction requested=true 2023-02-12 14:53:48,807 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:48,807 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:48,808 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:48,808 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,808 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:48,808 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:48,809 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,809 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:48,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:48,810 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,810 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:48,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:48,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:48,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:48,811 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a7e501[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:48,811 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:48,812 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,813 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,813 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:48,813 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:48,813 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:48,813 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,813 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:48,814 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,814 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:48,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:48,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:48,814 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:48,814 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:48,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:48,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:48,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:48,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:48,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:48,818 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:48,818 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:48,818 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:48,818 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,818 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:48,820 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,820 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,820 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:48,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:48,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:48,821 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:48,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,823 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:48,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:48,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:48,826 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:48,827 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-12 14:53:48,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:48,827 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:48,827 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,827 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:48,827 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:48,827 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,828 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,828 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,828 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:48,828 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:48,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:48,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:48,928 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:48,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:48,937 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:48,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:48,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,940 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:48,940 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:48,941 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:48,941 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:48,941 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,941 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,942 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:48,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:48,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:48,943 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:48,943 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:48,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:48,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,944 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,944 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:48,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:48,944 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:48,944 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:48,944 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:48,945 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:48,945 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:48,945 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:48,945 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:48,945 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:49,364 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509074fdd1865298a48a99a691b08b4f97cf8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201509074fdd1865298a48a99a691b08b4f97cf8 2023-02-12 14:53:49,774 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/b3619bb9f4be419eaea9aceb8467df28 2023-02-12 14:53:49,782 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/b3619bb9f4be419eaea9aceb8467df28 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/b3619bb9f4be419eaea9aceb8467df28 2023-02-12 14:53:49,787 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/b3619bb9f4be419eaea9aceb8467df28, entries=1, sequenceid=37, filesize=5.0 K 2023-02-12 14:53:49,789 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 844ms, sequenceid=37, compaction requested=true 2023-02-12 14:53:49,789 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:49,789 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:49,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:49,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:49,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:49,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:49,791 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,791 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:49,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:49,791 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,792 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:49,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:49,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:49,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@17e1efc8[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:49,793 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:49,793 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,797 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,797 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,797 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:49,797 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:49,797 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,797 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:49,797 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:49,799 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:49,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:49,799 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:49,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:49,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:49,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:49,802 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:49,802 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:49,802 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:49,802 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:49,802 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,802 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:49,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:49,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,804 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:49,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:49,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:49,804 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:49,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,805 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:49,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,808 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,808 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:49,808 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,808 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:49,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly because node does not exist (not an error) 2023-02-12 14:53:49,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:49,808 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:49,808 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,808 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:49,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:49,809 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,809 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,809 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,809 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:49,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:49,809 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:49,810 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:49,910 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:49,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:49,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:49,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthly was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:49,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,920 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:49,920 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:49,921 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' 2023-02-12 14:53:49,921 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthly', kicking off acquire phase on members. 2023-02-12 14:53:49,921 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,921 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,924 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,924 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:49,924 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:49,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:49,924 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:49,924 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:49,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:49,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,925 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthly is still present, removing 2023-02-12 14:53:49,925 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:49,925 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' with timeout 60000ms 2023-02-12 14:53:49,925 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:49,926 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'acquire' stage 2023-02-12 14:53:49,926 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:49,926 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:49,927 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:49,927 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. started... 2023-02-12 14:53:49,927 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 0384a0af3ce8daed2b5dc2a6da84edf7 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:50,346 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120288261b010c94e72ad65b85ecced8dee to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120288261b010c94e72ad65b85ecced8dee 2023-02-12 14:53:50,755 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1 2023-02-12 14:53:50,762 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1 2023-02-12 14:53:50,769 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1, entries=1, sequenceid=41, filesize=5.0 K 2023-02-12 14:53:50,770 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 0384a0af3ce8daed2b5dc2a6da84edf7 in 843ms, sequenceid=41, compaction requested=true 2023-02-12 14:53:50,770 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:50,770 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:50,770 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:50,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:50,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally acquired 2023-02-12 14:53:50,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthly) in zk 2023-02-12 14:53:50,772 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,772 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:50,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:50,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:50,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' on coordinator 2023-02-12 14:53:50,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6733ac90[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:50,774 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' starting 'in-barrier' execution. 2023-02-12 14:53:50,774 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,775 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,776 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' received 'reached' from coordinator. 2023-02-12 14:53:50,776 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' locally completed 2023-02-12 14:53:50,776 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:50,776 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,776 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:50,780 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,780 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' has notified controller of completion 2023-02-12 14:53:50,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:50,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:50,780 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:50,780 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthly' completed. 2023-02-12 14:53:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:50,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,784 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:50,784 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthly', counting down latch. Waiting for 0 more 2023-02-12 14:53:50,784 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthly' execution completed 2023-02-12 14:53:50,784 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:50,784 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:50,784 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,784 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:50,786 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,786 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:50,786 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:50,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:50,787 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:50,787 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:50,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,788 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:50,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,791 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,791 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:50,791 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,791 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:50,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthly' 2023-02-12 14:53:50,791 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:50,791 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:50,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:50,791 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:50,791 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,792 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:50,792 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,792 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,792 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,792 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthly'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:50,792 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:50,792 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:50,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:50,892 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:50,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:50,898 DEBUG [Listener at localhost/37945] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-12 14:53:50,901 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51856, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-12 14:53:50,902 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:50,914 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-12 14:53:50,915 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=12, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-12 14:53:50,917 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:53:50,917 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=12, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214230917 2023-02-12 14:53:50,918 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:53:50,922 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-12 14:53:50,922 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:50,922 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x423651f1 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:53:50,947 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@12117521, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:53:50,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,817 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103839c2fa9bd494ce28a85c3f43548a09a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160103839c2fa9bd494ce28a85c3f43548a09a 2023-02-12 14:53:51,818 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:51,838 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:53:51,839 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:58028, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:53:51,849 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:53:51,851 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:51872, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:53:51,884 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/36832687b2394360ac813e504b4688fb first=Optional[r3] last=Optional[r6] 2023-02-12 14:53:51,896 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/36832687b2394360ac813e504b4688fb}] 2023-02-12 14:53:51,912 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/36832687b2394360ac813e504b4688fb for inclusion in 0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:51,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-12 14:53:51,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:53:51,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:51,920 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/36832687b2394360ac813e504b4688fb to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3jh2a5gp71hktk88fself74p506h22vanbvsoo0ialesp6c0397jhrr0lpc16dai/family1/36832687b2394360ac813e504b4688fb 2023-02-12 14:53:51,924 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3jh2a5gp71hktk88fself74p506h22vanbvsoo0ialesp6c0397jhrr0lpc16dai/family1/36832687b2394360ac813e504b4688fb as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_ 2023-02-12 14:53:51,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3jh2a5gp71hktk88fself74p506h22vanbvsoo0ialesp6c0397jhrr0lpc16dai/family1/36832687b2394360ac813e504b4688fb into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_ - updating store file list. 2023-02-12 14:53:51,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_ into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:51,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3jh2a5gp71hktk88fself74p506h22vanbvsoo0ialesp6c0397jhrr0lpc16dai/family1/36832687b2394360ac813e504b4688fb into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_) 2023-02-12 14:53:51,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__3jh2a5gp71hktk88fself74p506h22vanbvsoo0ialesp6c0397jhrr0lpc16dai/family1/36832687b2394360ac813e504b4688fb 2023-02-12 14:53:51,933 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-12 14:53:51,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51357 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-12 14:53:51,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 0384a0af3ce8daed2b5dc2a6da84edf7/family1 is initiating minor compaction 2023-02-12 14:53:51,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@81f6ca0; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:53:51,942 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0384a0af3ce8daed2b5dc2a6da84edf7/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:51,943 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/098010b74de345f88e97251e8eca3d3f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/a050537be3044f08b06c3c125b411995, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/89b41e5ba0a5447f971263bba825b81a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/22544ede5bc24332a770cc327a76d202, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/2428093bad3b4a88b8641be164950798, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/3ecd02539aa3467da683f9107cb6f0a1, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/b3619bb9f4be419eaea9aceb8467df28, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp, totalSize=50.2 K 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 098010b74de345f88e97251e8eca3d3f, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting a050537be3044f08b06c3c125b411995, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 89b41e5ba0a5447f971263bba825b81a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting d20ba8986a0e4a4aa9a9e5c3e865e64c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 22544ede5bc24332a770cc327a76d202, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 2428093bad3b4a88b8641be164950798, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 3ecd02539aa3467da683f9107cb6f0a1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting b3619bb9f4be419eaea9aceb8467df28, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting eceb016e80ad40cd9e1a6d7c3f0ed2e1, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-12 14:53:51,944 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 26a9b847f906431e906cc92514a26ca9_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-12 14:53:51,956 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:51,958 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:53:51,963 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228ac81b98035454d2292278e6a81f1ba0b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151228ac81b98035454d2292278e6a81f1ba0b 2023-02-12 14:53:51,965 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317f5436f7baa44d648483de22a3efd109 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512317f5436f7baa44d648483de22a3efd109 2023-02-12 14:53:51,967 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101d502652d90e14739b2c3230d96688ab1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20160101d502652d90e14739b2c3230d96688ab1 2023-02-12 14:53:51,968 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601030b0b60a028ba446d8be95e0bc96115f4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201601030b0b60a028ba446d8be95e0bc96115f4 2023-02-12 14:53:51,969 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 4 expired mob files are deleted 2023-02-12 14:53:51,969 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-12 14:53:51,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,969 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,970 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:51,995 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0384a0af3ce8daed2b5dc2a6da84edf7#family1#compaction#10 average throughput is 0.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:53:52,425 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/5b2ab5b2ad0c4c92bccb68db1c518d0c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/5b2ab5b2ad0c4c92bccb68db1c518d0c 2023-02-12 14:53:52,440 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 0384a0af3ce8daed2b5dc2a6da84edf7/family1 of 0384a0af3ce8daed2b5dc2a6da84edf7 into 5b2ab5b2ad0c4c92bccb68db1c518d0c(size=6.6 K), total size for store is 182.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:53:52,440 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:52,441 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7., storeName=family1, fileCount=10, fileSize=50.2 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.5 K), priority=2, time=1676213631941; duration=0sec 2023-02-12 14:53:52,442 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:53:52,460 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/098010b74de345f88e97251e8eca3d3f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/a050537be3044f08b06c3c125b411995, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/89b41e5ba0a5447f971263bba825b81a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/22544ede5bc24332a770cc327a76d202, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/2428093bad3b4a88b8641be164950798, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/3ecd02539aa3467da683f9107cb6f0a1, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/b3619bb9f4be419eaea9aceb8467df28, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_] to archive 2023-02-12 14:53:52,461 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:53:52,464 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/098010b74de345f88e97251e8eca3d3f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/098010b74de345f88e97251e8eca3d3f 2023-02-12 14:53:52,466 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/a050537be3044f08b06c3c125b411995 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/a050537be3044f08b06c3c125b411995 2023-02-12 14:53:52,467 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/89b41e5ba0a5447f971263bba825b81a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/89b41e5ba0a5447f971263bba825b81a 2023-02-12 14:53:52,468 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/d20ba8986a0e4a4aa9a9e5c3e865e64c 2023-02-12 14:53:52,470 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/22544ede5bc24332a770cc327a76d202 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/22544ede5bc24332a770cc327a76d202 2023-02-12 14:53:52,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/2428093bad3b4a88b8641be164950798 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/2428093bad3b4a88b8641be164950798 2023-02-12 14:53:52,473 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/3ecd02539aa3467da683f9107cb6f0a1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/3ecd02539aa3467da683f9107cb6f0a1 2023-02-12 14:53:52,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/b3619bb9f4be419eaea9aceb8467df28 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/b3619bb9f4be419eaea9aceb8467df28 2023-02-12 14:53:52,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/eceb016e80ad40cd9e1a6d7c3f0ed2e1 2023-02-12 14:53:52,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/26a9b847f906431e906cc92514a26ca9_SeqId_44_ 2023-02-12 14:53:52,856 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512058fcc4994c159456d90ba5111f5960e07 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e201512058fcc4994c159456d90ba5111f5960e07 2023-02-12 14:53:52,858 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:52,888 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ada30437290d4433aba690324b926462 first=Optional[r00] last=Optional[r2] 2023-02-12 14:53:52,889 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ada30437290d4433aba690324b926462}] 2023-02-12 14:53:52,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ada30437290d4433aba690324b926462 for inclusion in 0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:52,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-12 14:53:52,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:53:52,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:52,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/ada30437290d4433aba690324b926462 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__or95odefcrofideu4b8c5cdg441408hi71ohsmcj9th95ha1ioj6r6l7dq7p9tk7/family1/ada30437290d4433aba690324b926462 2023-02-12 14:53:52,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__or95odefcrofideu4b8c5cdg441408hi71ohsmcj9th95ha1ioj6r6l7dq7p9tk7/family1/ada30437290d4433aba690324b926462 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_ 2023-02-12 14:53:52,901 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__or95odefcrofideu4b8c5cdg441408hi71ohsmcj9th95ha1ioj6r6l7dq7p9tk7/family1/ada30437290d4433aba690324b926462 into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_ - updating store file list. 2023-02-12 14:53:52,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_ into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:52,908 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__or95odefcrofideu4b8c5cdg441408hi71ohsmcj9th95ha1ioj6r6l7dq7p9tk7/family1/ada30437290d4433aba690324b926462 into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_) 2023-02-12 14:53:52,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__or95odefcrofideu4b8c5cdg441408hi71ohsmcj9th95ha1ioj6r6l7dq7p9tk7/family1/ada30437290d4433aba690324b926462 2023-02-12 14:53:52,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-12 14:53:52,910 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 367249 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-12 14:53:52,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 0384a0af3ce8daed2b5dc2a6da84edf7/family1 is initiating minor compaction (all files) 2023-02-12 14:53:52,911 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@42cd69f7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:53:52,911 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 0384a0af3ce8daed2b5dc2a6da84edf7/family1 in testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:52,911 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c9b044972b7a40ffac92814d3a926e39, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/5b2ab5b2ad0c4c92bccb68db1c518d0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp, totalSize=358.6 K 2023-02-12 14:53:52,912 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting c9b044972b7a40ffac92814d3a926e39, keycount=1000, bloomtype=ROW, size=175.8 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840591 2023-02-12 14:53:52,913 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 5b2ab5b2ad0c4c92bccb68db1c518d0c, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240591 2023-02-12 14:53:52,914 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting c7a62606b06046db8cd4a6953e36e345_SeqId_47_, keycount=1002, bloomtype=ROW, size=176.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840591 2023-02-12 14:53:52,914 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:52,917 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:53:52,919 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a083ed1d440a416185d5fee3670f8b80 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151130a083ed1d440a416185d5fee3670f8b80 2023-02-12 14:53:52,921 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201777f847aa08a465fa599e25efd56f6c1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151201777f847aa08a465fa599e25efd56f6c1 2023-02-12 14:53:52,923 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205fc9befbeb5dd4c8d8ed2030efe6a9e3b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151205fc9befbeb5dd4c8d8ed2030efe6a9e3b 2023-02-12 14:53:52,923 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 3 expired mob files are deleted 2023-02-12 14:53:52,923 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-12 14:53:52,924 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:52,924 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:52,924 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:53:52,925 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=12, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-12 14:53:52,928 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 2.0180 sec 2023-02-12 14:53:52,964 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0384a0af3ce8daed2b5dc2a6da84edf7#family1#compaction#11 average throughput is 4.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:53:53,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthly from 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-12 14:53:53,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=13, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:53,494 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-12 14:53:53,495 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000001 2023-02-12 14:53:53,495 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/.tabledesc/.tableinfo.0000000002 2023-02-12 14:53:53,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly}] 2023-02-12 14:53:53,514 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, REOPEN/MOVE}] 2023-02-12 14:53:53,516 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, REOPEN/MOVE 2023-02-12 14:53:53,517 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0384a0af3ce8daed2b5dc2a6da84edf7, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:53,517 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676213633517"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213633517"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213633517"}]},"ts":"1676213633517"} 2023-02-12 14:53:53,519 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=16, ppid=15, state=RUNNABLE; CloseRegionProcedure 0384a0af3ce8daed2b5dc2a6da84edf7, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:53:53,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:53,679 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0384a0af3ce8daed2b5dc2a6da84edf7, disabling compactions & flushes 2023-02-12 14:53:53,679 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:53,797 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/.tmp/family1/cd1023060b224e83acb875e08696e96c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/cd1023060b224e83acb875e08696e96c 2023-02-12 14:53:53,804 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 0384a0af3ce8daed2b5dc2a6da84edf7/family1 of 0384a0af3ce8daed2b5dc2a6da84edf7 into cd1023060b224e83acb875e08696e96c(size=177.2 K), total size for store is 177.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:53:53,804 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:53,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:53,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:53,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. after waiting 0 ms 2023-02-12 14:53:53,804 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7., storeName=family1, fileCount=3, fileSize=358.6 K (175.8 K, 6.6 K, 176.2 K), priority=2, time=1676213632910; duration=0sec 2023-02-12 14:53:53,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:53,804 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:53:53,807 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c9b044972b7a40ffac92814d3a926e39, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/5b2ab5b2ad0c4c92bccb68db1c518d0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_] to archive 2023-02-12 14:53:53,808 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:53:53,810 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c9b044972b7a40ffac92814d3a926e39 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c9b044972b7a40ffac92814d3a926e39 2023-02-12 14:53:53,811 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/5b2ab5b2ad0c4c92bccb68db1c518d0c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/5b2ab5b2ad0c4c92bccb68db1c518d0c 2023-02-12 14:53:53,812 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/c7a62606b06046db8cd4a6953e36e345_SeqId_47_ 2023-02-12 14:53:53,819 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-12 14:53:53,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:53,820 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0384a0af3ce8daed2b5dc2a6da84edf7: Waiting for close lock at 1676213633679Running coprocessor pre-close hooks at 1676213633679Disabling compacts and flushes for region at 1676213633679Disabling writes for close at 1676213633804 (+125 ms)Writing region close event to WAL at 1676213633815 (+11 ms)Running coprocessor post-close hooks at 1676213633819 (+4 ms)Closed at 1676213633820 (+1 ms) 2023-02-12 14:53:53,820 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 0384a0af3ce8daed2b5dc2a6da84edf7 to self. 2023-02-12 14:53:53,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:53,823 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0384a0af3ce8daed2b5dc2a6da84edf7, regionState=CLOSED 2023-02-12 14:53:53,823 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676213633823"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213633823"}]},"ts":"1676213633823"} 2023-02-12 14:53:53,827 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=15 2023-02-12 14:53:53,827 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=15, state=SUCCESS; CloseRegionProcedure 0384a0af3ce8daed2b5dc2a6da84edf7, server=jenkins-hbase4.apache.org,45319,1676213617001 in 306 msec 2023-02-12 14:53:53,828 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=14, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=true 2023-02-12 14:53:53,978 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0384a0af3ce8daed2b5dc2a6da84edf7, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:53,979 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676213633978"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213633978"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213633978"}]},"ts":"1676213633978"} 2023-02-12 14:53:53,981 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 0384a0af3ce8daed2b5dc2a6da84edf7, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:53:54,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:54,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0384a0af3ce8daed2b5dc2a6da84edf7, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:53:54,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthly 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:54,138 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:54,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:54,139 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:54,140 INFO [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:54,142 DEBUG [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:54,142 DEBUG [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:54,142 INFO [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0384a0af3ce8daed2b5dc2a6da84edf7 columnFamilyName family1 2023-02-12 14:53:54,150 DEBUG [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/cd1023060b224e83acb875e08696e96c 2023-02-12 14:53:54,150 INFO [StoreOpener-0384a0af3ce8daed2b5dc2a6da84edf7-1] regionserver.HStore(310): Store=0384a0af3ce8daed2b5dc2a6da84edf7/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:54,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:54,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:54,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:53:54,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 0384a0af3ce8daed2b5dc2a6da84edf7; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63679970, jitterRate=-0.051094502210617065}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:53:54,157 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 0384a0af3ce8daed2b5dc2a6da84edf7: Running coprocessor pre-open hook at 1676213634139Writing region info on filesystem at 1676213634139Initializing all the Stores at 1676213634140 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676213634140Cleaning up temporary data from old regions at 1676213634153 (+13 ms)Cleaning up detritus from prior splits at 1676213634154 (+1 ms)Running coprocessor post-open hooks at 1676213634157 (+3 ms)Region opened successfully at 1676213634157 2023-02-12 14:53:54,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7., pid=17, masterSystemTime=1676213634134 2023-02-12 14:53:54,160 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:54,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:53:54,161 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=0384a0af3ce8daed2b5dc2a6da84edf7, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:54,161 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7.","families":{"info":[{"qualifier":"regioninfo","vlen":80,"tag":[],"timestamp":"1676213634160"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213634160"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213634160"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213634160"}]},"ts":"1676213634160"} 2023-02-12 14:53:54,165 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-02-12 14:53:54,165 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 0384a0af3ce8daed2b5dc2a6da84edf7, server=jenkins-hbase4.apache.org,45319,1676213617001 in 182 msec 2023-02-12 14:53:54,167 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=14 2023-02-12 14:53:54,167 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=14, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly, region=0384a0af3ce8daed2b5dc2a6da84edf7, REOPEN/MOVE in 651 msec 2023-02-12 14:53:54,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-12 14:53:54,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 670 msec 2023-02-12 14:53:54,173 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=13, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthly in 1.1010 sec 2023-02-12 14:53:54,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=13 2023-02-12 14:53:54,183 INFO [Listener at localhost/37945] compactions.TestMobCompactor(278): alter status finished 2023-02-12 14:53:54,184 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthly 2023-02-12 14:53:54,185 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-12 14:53:54,186 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=18, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-12 14:53:54,187 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:53:54,187 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=18, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214234187 2023-02-12 14:53:54,188 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:53:54,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthly' and column 'family1' 2023-02-12 14:53:54,189 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:54,189 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x341f9cde to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:53:54,199 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5c132845, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:53:54,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:54,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:54,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:54,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:54,199 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:55,034 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112818c77ec93bea4f3285dc67edc8b0488b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e2015112818c77ec93bea4f3285dc67edc8b0488b 2023-02-12 14:53:55,035 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:55,048 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:53:55,050 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38886, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:53:55,058 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:53:55,060 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:36896, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:53:55,071 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ada18d4da5c140bf9f847e443804f9d0 first=Optional[r05] last=Optional[r8] 2023-02-12 14:53:55,072 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ada18d4da5c140bf9f847e443804f9d0}] 2023-02-12 14:53:55,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ada18d4da5c140bf9f847e443804f9d0 for inclusion in 0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:55,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-12 14:53:55,081 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:53:55,082 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 0384a0af3ce8daed2b5dc2a6da84edf7: 2023-02-12 14:53:55,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/ada18d4da5c140bf9f847e443804f9d0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__21pp67r8u7ouftmut4107ka0re661ofeun8f8qe7c15kucb8tmavkv2i3robet3v/family1/ada18d4da5c140bf9f847e443804f9d0 2023-02-12 14:53:55,086 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__21pp67r8u7ouftmut4107ka0re661ofeun8f8qe7c15kucb8tmavkv2i3robet3v/family1/ada18d4da5c140bf9f847e443804f9d0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/cc8d4a20671240e2bed46ef39816cf14_SeqId_53_ 2023-02-12 14:53:55,086 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__21pp67r8u7ouftmut4107ka0re661ofeun8f8qe7c15kucb8tmavkv2i3robet3v/family1/ada18d4da5c140bf9f847e443804f9d0 into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/cc8d4a20671240e2bed46ef39816cf14_SeqId_53_ - updating store file list. 2023-02-12 14:53:55,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/cc8d4a20671240e2bed46ef39816cf14_SeqId_53_ into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 2023-02-12 14:53:55,091 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__21pp67r8u7ouftmut4107ka0re661ofeun8f8qe7c15kucb8tmavkv2i3robet3v/family1/ada18d4da5c140bf9f847e443804f9d0 into 0384a0af3ce8daed2b5dc2a6da84edf7/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/family1/cc8d4a20671240e2bed46ef39816cf14_SeqId_53_) 2023-02-12 14:53:55,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthly__21pp67r8u7ouftmut4107ka0re661ofeun8f8qe7c15kucb8tmavkv2i3robet3v/family1/ada18d4da5c140bf9f847e443804f9d0 2023-02-12 14:53:55,094 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:55,095 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:53:55,097 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120288261b010c94e72ad65b85ecced8dee to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151120288261b010c94e72ad65b85ecced8dee 2023-02-12 14:53:55,098 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128b20355a1dcf549f0b281418084bd9637 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/5cf2233aabbe1315e9fba5c3199b4c19/family1/d41d8cd98f00b204e9800998ecf8427e20151128b20355a1dcf549f0b281418084bd9637 2023-02-12 14:53:55,099 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthly 2 expired mob files are deleted 2023-02-12 14:53:55,099 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-12 14:53:55,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:55,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:55,099 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthly' and column='family1' 2023-02-12 14:53:55,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthly' column='family1' 2023-02-12 14:53:55,099 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:53:55,100 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=18, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE 2023-02-12 14:53:55,102 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=18, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthly.mobLock, type=EXCLUSIVE in 915 msec 2023-02-12 14:53:55,180 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthly Thread=247 (was 212) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45158 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x649d067d-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-1-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3938806-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45292 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45196 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45226 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: htable-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45302 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45384 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45328 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45382 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x423651f1-SendThread(127.0.0.1:61565) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x341f9cde-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase4:45319-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x649d067d-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x423651f1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/657284103.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-4-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x341f9cde sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/657284103.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3938806-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: MobCompactor-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45338 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-254133399_15 at /127.0.0.1:45210 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45208 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45296 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3938806-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x341f9cde-SendThread(127.0.0.1:61565) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45264 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hfile_cleaner-dir-scan-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45188 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x423651f1-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:45334 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=588 (was 475) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=271 (was 294), ProcessCount=168 (was 168), AvailableMemoryMB=4733 (was 4867) 2023-02-12 14:53:55,192 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=247, OpenFileDescriptor=588, MaxFileDescriptor=60000, SystemLoadAverage=271, ProcessCount=168, AvailableMemoryMB=4732 2023-02-12 14:53:55,194 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-12 14:53:55,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:55,197 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:53:55,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithMonthlyPolicy" procId is: 19 2023-02-12 14:53:55,198 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:53:55,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 14:53:55,200 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:55,200 DEBUG [HFileArchiver-3] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea empty. 2023-02-12 14:53:55,201 DEBUG [HFileArchiver-3] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:55,201 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithMonthlyPolicy regions 2023-02-12 14:53:55,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 14:53:55,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 14:53:55,612 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-12 14:53:55,613 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 618f7278d7d2801ef6c08ce41da650ea, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:53:55,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 14:53:56,020 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:56,020 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 618f7278d7d2801ef6c08ce41da650ea, disabling compactions & flushes 2023-02-12 14:53:56,020 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:56,020 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:56,020 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. after waiting 0 ms 2023-02-12 14:53:56,020 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:56,020 INFO [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:56,020 DEBUG [RegionOpenAndInit-testMajorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 618f7278d7d2801ef6c08ce41da650ea: Waiting for close lock at 1676213636020Disabling compacts and flushes for region at 1676213636020Disabling writes for close at 1676213636020Writing region close event to WAL at 1676213636020Closed at 1676213636020 2023-02-12 14:53:56,023 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:53:56,024 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676213636024"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213636024"}]},"ts":"1676213636024"} 2023-02-12 14:53:56,026 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:53:56,027 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:53:56,028 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213636028"}]},"ts":"1676213636028"} 2023-02-12 14:53:56,029 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-12 14:53:56,032 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=618f7278d7d2801ef6c08ce41da650ea, ASSIGN}] 2023-02-12 14:53:56,033 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=618f7278d7d2801ef6c08ce41da650ea, ASSIGN 2023-02-12 14:53:56,034 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=618f7278d7d2801ef6c08ce41da650ea, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:53:56,185 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=618f7278d7d2801ef6c08ce41da650ea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:56,185 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676213636185"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213636185"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213636185"}]},"ts":"1676213636185"} 2023-02-12 14:53:56,187 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=21, ppid=20, state=RUNNABLE; OpenRegionProcedure 618f7278d7d2801ef6c08ce41da650ea, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:53:56,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 14:53:56,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:56,343 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 618f7278d7d2801ef6c08ce41da650ea, NAME => 'testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:53:56,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithMonthlyPolicy 618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:56,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:53:56,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:56,344 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:56,345 INFO [StoreOpener-618f7278d7d2801ef6c08ce41da650ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:56,347 DEBUG [StoreOpener-618f7278d7d2801ef6c08ce41da650ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:53:56,347 DEBUG [StoreOpener-618f7278d7d2801ef6c08ce41da650ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:53:56,347 INFO [StoreOpener-618f7278d7d2801ef6c08ce41da650ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 618f7278d7d2801ef6c08ce41da650ea columnFamilyName family1 2023-02-12 14:53:56,349 INFO [StoreOpener-618f7278d7d2801ef6c08ce41da650ea-1] regionserver.HStore(310): Store=618f7278d7d2801ef6c08ce41da650ea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:53:56,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:56,350 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:56,353 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:53:56,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:53:56,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 618f7278d7d2801ef6c08ce41da650ea; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59328427, jitterRate=-0.11593754589557648}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:53:56,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 618f7278d7d2801ef6c08ce41da650ea: Running coprocessor pre-open hook at 1676213636344Writing region info on filesystem at 1676213636344Initializing all the Stores at 1676213636345 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676213636345Cleaning up temporary data from old regions at 1676213636351 (+6 ms)Cleaning up detritus from prior splits at 1676213636351Running coprocessor post-open hooks at 1676213636355 (+4 ms)Region opened successfully at 1676213636355 2023-02-12 14:53:56,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea., pid=21, masterSystemTime=1676213636339 2023-02-12 14:53:56,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:56,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:56,359 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=618f7278d7d2801ef6c08ce41da650ea, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:56,359 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676213636359"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213636359"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213636359"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213636359"}]},"ts":"1676213636359"} 2023-02-12 14:53:56,363 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=20 2023-02-12 14:53:56,363 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=20, state=SUCCESS; OpenRegionProcedure 618f7278d7d2801ef6c08ce41da650ea, server=jenkins-hbase4.apache.org,45319,1676213617001 in 174 msec 2023-02-12 14:53:56,365 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=20, resume processing ppid=19 2023-02-12 14:53:56,365 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithMonthlyPolicy, region=618f7278d7d2801ef6c08ce41da650ea, ASSIGN in 331 msec 2023-02-12 14:53:56,366 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:53:56,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213636366"}]},"ts":"1676213636366"} 2023-02-12 14:53:56,367 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-12 14:53:56,371 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=19, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:53:56,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithMonthlyPolicy in 1.1760 sec 2023-02-12 14:53:57,304 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-12 14:53:57,305 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithMonthlyPolicy, procId: 19 completed 2023-02-12 14:53:57,340 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:57,345 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,346 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:57,346 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:57,346 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:53:57,346 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:53:57,347 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,347 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,348 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:57,348 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:57,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:57,349 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,349 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:57,349 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:57,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,352 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,352 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:53:57,352 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:57,353 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:53:57,353 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:57,353 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:57,353 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:57,353 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:53:57,354 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-12 14:53:57,426 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511303686c35f1e7740c9ae3d13f0699456dd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511303686c35f1e7740c9ae3d13f0699456dd 2023-02-12 14:53:57,832 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/a114d7926fe9418c833680829967390e 2023-02-12 14:53:57,839 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/a114d7926fe9418c833680829967390e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a114d7926fe9418c833680829967390e 2023-02-12 14:53:57,844 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a114d7926fe9418c833680829967390e, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-12 14:53:57,845 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 492ms, sequenceid=5, compaction requested=false 2023-02-12 14:53:57,845 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:57,846 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:53:57,846 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:57,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:57,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:57,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:53:57,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:53:57,848 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:57,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,849 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:57,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:57,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:53:57,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ff354cd[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:57,851 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:53:57,851 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,852 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,852 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:53:57,852 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:53:57,852 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,852 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:57,852 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:57,855 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,855 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:53:57,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:57,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:57,855 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:57,855 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:53:57,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:57,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:57,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:57,859 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:53:57,859 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:53:57,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:57,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:57,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,859 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:57,860 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,861 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,861 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:57,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:57,861 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:57,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,863 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:57,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:57,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,865 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:57,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:57,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:57,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:57,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:57,866 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,867 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:57,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:53:57,867 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:57,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:57,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:57,967 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:57,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:57,974 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:57,975 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:57,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,976 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:57,976 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:57,976 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:53:57,976 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:53:57,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,978 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:57,978 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:57,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,979 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:57,979 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:57,979 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:57,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,979 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:53:57,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:57,980 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:53:57,980 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:57,980 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:53:57,981 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:57,981 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:57,981 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:57,981 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:53:57,981 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-12 14:53:58,407 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283fa22787acbd462bbf96731275332263 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511283fa22787acbd462bbf96731275332263 2023-02-12 14:53:58,817 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/ccb2f71ce87446a2a5139f193725bc7b 2023-02-12 14:53:58,825 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/ccb2f71ce87446a2a5139f193725bc7b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/ccb2f71ce87446a2a5139f193725bc7b 2023-02-12 14:53:58,833 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/ccb2f71ce87446a2a5139f193725bc7b, entries=1, sequenceid=9, filesize=5.0 K 2023-02-12 14:53:58,834 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 853ms, sequenceid=9, compaction requested=false 2023-02-12 14:53:58,834 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:53:58,834 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:58,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:58,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:58,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:53:58,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:53:58,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:58,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:58,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:58,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:58,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:53:58,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77442fdf[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:58,840 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:53:58,840 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,844 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:53:58,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:58,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:58,846 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:58,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:58,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:53:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:58,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:53:58,851 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:53:58,851 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:58,851 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:58,851 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,851 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:58,854 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,854 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:58,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,854 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:58,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:58,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,858 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:58,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:58,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:58,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:58,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:58,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:58,867 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:58,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,867 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:58,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,867 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,868 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,868 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,868 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:58,868 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:58,868 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:58,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:58,969 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:58,970 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:58,975 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:58,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:58,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,982 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:58,982 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:58,982 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:53:58,983 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:53:58,984 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,984 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,988 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:58,988 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,988 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:58,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:58,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,988 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:58,989 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:58,989 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:58,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,989 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,990 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:53:58,990 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:53:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:58,990 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:53:59,001 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:59,001 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:59,001 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:59,001 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:53:59,001 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:53:59,359 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:53:59,425 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512014f6d6a4170524b8d8c22f615b54b60cc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512014f6d6a4170524b8d8c22f615b54b60cc 2023-02-12 14:53:59,832 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/6782d10a3ff94510a5c9544fe9181620 2023-02-12 14:53:59,839 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/6782d10a3ff94510a5c9544fe9181620 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6782d10a3ff94510a5c9544fe9181620 2023-02-12 14:53:59,844 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6782d10a3ff94510a5c9544fe9181620, entries=1, sequenceid=13, filesize=4.9 K 2023-02-12 14:53:59,845 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 844ms, sequenceid=13, compaction requested=true 2023-02-12 14:53:59,845 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:53:59,845 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,845 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:53:59,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:53:59,847 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,847 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:59,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:59,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:53:59,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:59,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:59,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:59,849 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:53:59,850 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f430150[Count = 0] remaining members to acquire global barrier 2023-02-12 14:53:59,850 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:53:59,850 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,852 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,852 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:53:59,852 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:53:59,852 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:53:59,852 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,852 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:53:59,853 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:53:59,853 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,853 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:59,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:59,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:59,853 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:53:59,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:59,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:59,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:59,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:53:59,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:53:59,856 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:53:59,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:53:59,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:53:59,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,856 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:53:59,858 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,858 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,858 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:59,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:53:59,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:53:59,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:59,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:59,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:53:59,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:53:59,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,860 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:59,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:59,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:53:59,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:59,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,863 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:53:59,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:59,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:53:59,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:53:59,864 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:53:59,864 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,864 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:53:59,864 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:53:59,868 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:53:59,964 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:53:59,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:53:59,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:53:59,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:53:59,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,979 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:53:59,979 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:59,980 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:53:59,980 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:53:59,980 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,980 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,982 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,982 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:53:59,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:53:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:53:59,983 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:53:59,983 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:53:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:53:59,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:53:59,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:53:59,987 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:53:59,987 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:53:59,988 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:53:59,988 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:53:59,988 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:53:59,988 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:53:59,988 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:53:59,988 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:00,409 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512053e1efef7cec6424e87c663b776274753 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512053e1efef7cec6424e87c663b776274753 2023-02-12 14:54:00,816 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/cff46623305f459a90f2958b8807635e 2023-02-12 14:54:00,831 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/cff46623305f459a90f2958b8807635e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/cff46623305f459a90f2958b8807635e 2023-02-12 14:54:00,839 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/cff46623305f459a90f2958b8807635e, entries=1, sequenceid=17, filesize=4.9 K 2023-02-12 14:54:00,841 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 853ms, sequenceid=17, compaction requested=true 2023-02-12 14:54:00,841 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:00,841 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:54:00,841 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:54:00,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:00,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:00,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:00,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:00,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:00,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:00,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:54:00,845 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15ee7704[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:00,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:00,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,846 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,846 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:00,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:54:00,846 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:00,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:00,849 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:54:00,849 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:00,849 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:54:00,849 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:00,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:00,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:00,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:00,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:00,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:00,856 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:00,856 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:54:00,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:00,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:00,856 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,856 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:00,858 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,858 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:00,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:00,858 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:00,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:00,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:00,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:00,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:00,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,861 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:00,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,863 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,864 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:00,864 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:00,864 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:54:00,864 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:00,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:00,864 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:00,864 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:00,864 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:00,865 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:00,865 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:00,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:00,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:00,965 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:00,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:00,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:00,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:00,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,977 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:00,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:00,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:54:00,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:00,978 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,978 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,979 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,979 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:00,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:00,979 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:00,980 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:00,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:00,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,981 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:54:00,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:00,984 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:54:00,984 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:00,984 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:54:00,987 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:00,987 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:00,987 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:00,987 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:54:00,988 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:01,413 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289ae5815d5b934771ae4fa5ca25e7a94f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512289ae5815d5b934771ae4fa5ca25e7a94f 2023-02-12 14:54:01,822 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/a318102869704ee08299e85b78a8e775 2023-02-12 14:54:01,829 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/a318102869704ee08299e85b78a8e775 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a318102869704ee08299e85b78a8e775 2023-02-12 14:54:01,834 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a318102869704ee08299e85b78a8e775, entries=1, sequenceid=21, filesize=4.9 K 2023-02-12 14:54:01,835 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 847ms, sequenceid=21, compaction requested=true 2023-02-12 14:54:01,835 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:01,835 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:01,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:01,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:01,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:54:01,836 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:54:01,837 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,837 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:01,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:01,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,838 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:01,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:01,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:01,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:01,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:54:01,840 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@40ee0c48[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:01,840 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:01,840 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,842 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:54:01,842 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:01,842 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,842 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:54:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:01,843 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,843 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:54:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:01,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:01,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:01,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:01,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:01,847 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:01,847 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:54:01,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:01,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:01,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,847 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:01,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,851 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:01,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:01,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:01,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:01,852 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:01,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:01,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,853 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:01,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:01,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:01,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:01,857 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:01,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:01,857 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:01,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:01,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,857 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,857 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:01,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:01,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:01,858 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:54:01,858 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:01,958 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:01,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:01,964 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:01,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:01,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,966 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:01,966 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:01,967 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:54:01,967 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:01,967 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,967 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,969 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,969 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:01,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:01,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:01,969 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:01,969 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:01,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,970 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,970 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,970 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:54:01,970 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:01,971 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:54:01,971 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:01,971 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:54:01,971 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:01,971 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:01,971 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:01,971 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:54:01,972 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:01,997 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c4eb8d0f7d0f4d4dad2b0afb87077f10 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c4eb8d0f7d0f4d4dad2b0afb87077f10 2023-02-12 14:54:02,406 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/06b4a08528b041e1844a9c0a8ba2db8f 2023-02-12 14:54:02,412 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/06b4a08528b041e1844a9c0a8ba2db8f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/06b4a08528b041e1844a9c0a8ba2db8f 2023-02-12 14:54:02,417 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/06b4a08528b041e1844a9c0a8ba2db8f, entries=1, sequenceid=25, filesize=4.9 K 2023-02-12 14:54:02,418 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 446ms, sequenceid=25, compaction requested=true 2023-02-12 14:54:02,418 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:02,419 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:54:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:54:02,420 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,420 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:02,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:02,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:02,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:02,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:02,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:54:02,422 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1602f95d[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:02,422 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:02,422 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,423 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,423 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:02,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:54:02,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:02,424 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,424 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:02,426 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:54:02,426 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,426 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:02,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:02,426 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:54:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:02,429 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:02,429 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:54:02,429 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:02,429 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:02,429 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,429 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:02,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:02,432 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:02,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:02,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,433 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,435 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:02,435 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:02,436 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:02,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:02,435 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:02,436 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,436 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:02,436 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:02,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:02,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:02,437 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:54:02,536 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:02,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:02,542 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:02,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:02,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,544 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:02,544 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:02,544 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:54:02,544 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:02,545 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,545 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,546 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,546 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:02,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:02,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:02,546 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,546 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:02,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:02,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:54:02,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,547 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:54:02,547 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:02,548 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:54:02,548 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:02,548 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:02,548 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:02,548 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:54:02,548 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:02,562 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010156ce2e1ed11045a8921880aff87a328e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010156ce2e1ed11045a8921880aff87a328e 2023-02-12 14:54:02,567 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/f1054c48bac741f1851c4c53c416adc2 2023-02-12 14:54:02,572 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/f1054c48bac741f1851c4c53c416adc2 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f1054c48bac741f1851c4c53c416adc2 2023-02-12 14:54:02,577 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f1054c48bac741f1851c4c53c416adc2, entries=1, sequenceid=29, filesize=4.9 K 2023-02-12 14:54:02,578 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 30ms, sequenceid=29, compaction requested=true 2023-02-12 14:54:02,578 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:02,578 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:02,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:02,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:02,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:54:02,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:54:02,580 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,580 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:02,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:02,581 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,581 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:02,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:02,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:02,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:02,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:54:02,583 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@613c334c[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:02,583 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:02,583 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,584 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,584 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:02,584 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:54:02,584 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:02,584 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,584 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:02,585 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:54:02,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:02,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:02,586 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:02,586 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:54:02,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:02,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:02,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:02,589 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:02,589 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:54:02,589 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:02,589 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:02,589 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,589 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:02,590 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,590 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:02,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,591 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:02,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:02,591 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:02,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:02,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,592 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:02,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,595 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,595 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:02,595 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:02,596 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,596 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:02,596 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,596 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,596 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,596 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:02,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:02,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:02,596 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:02,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:54:02,596 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:02,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:02,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:02,597 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:02,697 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:02,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:02,705 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:02,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:02,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,707 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:02,707 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:02,707 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:54:02,708 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:02,708 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,708 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,709 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:02,709 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:02,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:02,710 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:02,710 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:02,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:02,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,712 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:54:02,712 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:54:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:54:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:02,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:02,713 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:02,714 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:54:02,714 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:03,139 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103367a768999384938bd9847b8c389fe7c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103367a768999384938bd9847b8c389fe7c 2023-02-12 14:54:03,145 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/d8e67045ba8546bea7a44f095b87f56f 2023-02-12 14:54:03,153 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/d8e67045ba8546bea7a44f095b87f56f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/d8e67045ba8546bea7a44f095b87f56f 2023-02-12 14:54:03,158 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/d8e67045ba8546bea7a44f095b87f56f, entries=1, sequenceid=33, filesize=4.9 K 2023-02-12 14:54:03,159 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 445ms, sequenceid=33, compaction requested=true 2023-02-12 14:54:03,160 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:03,160 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:03,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:03,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:03,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:54:03,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:54:03,162 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,162 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:03,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:03,162 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,162 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:03,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:03,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:03,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:03,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:54:03,164 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78c3e38[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:03,164 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:03,164 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,165 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:54:03,165 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:03,165 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,165 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:03,169 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:54:03,169 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,169 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:03,169 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:54:03,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:03,173 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:03,173 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:54:03,173 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:03,173 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:03,173 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,173 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:03,174 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,174 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:03,174 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:03,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:03,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:03,175 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,177 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:03,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:03,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:03,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:03,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:03,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:03,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,179 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:03,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,180 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:03,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:03,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:03,180 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:03,180 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,180 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,180 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,280 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:03,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:03,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:03,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:03,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,289 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:03,289 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:03,289 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:54:03,289 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:03,290 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,290 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,291 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,291 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:03,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:03,291 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:03,291 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:03,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:03,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:54:03,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:03,292 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:54:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:54:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:03,293 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:03,293 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:03,293 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:54:03,293 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:03,709 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509079de2b516f50a407292919124d60ecc24 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201509079de2b516f50a407292919124d60ecc24 2023-02-12 14:54:04,116 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/043a2d82369340cd9a08cae04567cc41 2023-02-12 14:54:04,123 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/043a2d82369340cd9a08cae04567cc41 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/043a2d82369340cd9a08cae04567cc41 2023-02-12 14:54:04,128 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/043a2d82369340cd9a08cae04567cc41, entries=1, sequenceid=37, filesize=4.9 K 2023-02-12 14:54:04,130 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 837ms, sequenceid=37, compaction requested=true 2023-02-12 14:54:04,130 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:04,130 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:04,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:04,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:04,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:54:04,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:54:04,132 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,132 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:04,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:04,132 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,132 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:04,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:04,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:04,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:04,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:54:04,134 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@37b0eecd[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:04,134 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:04,134 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,135 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:54:04,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:04,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:54:04,137 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:04,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:04,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:04,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:54:04,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:04,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:04,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:04,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:04,141 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:54:04,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:04,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:04,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,141 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:04,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:04,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:04,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:04,145 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:04,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:04,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,146 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:04,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:04,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:04,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:04,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:04,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:54:04,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:04,149 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:04,149 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:04,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,149 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,149 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,149 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:04,150 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:04,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:04,150 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,150 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:04,150 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,250 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:04,251 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:04,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:04,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:04,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,258 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:04,258 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:04,259 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:54:04,259 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:04,260 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,260 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,261 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:04,261 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:04,261 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:04,261 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:04,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,262 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:04,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,262 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,263 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:54:04,263 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:04,263 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:54:04,263 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:04,264 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:54:04,265 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:04,265 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:04,265 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:04,265 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. started... 2023-02-12 14:54:04,265 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 618f7278d7d2801ef6c08ce41da650ea 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:04,694 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120249eef75af404c6b9f7741c1e40b9d01 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120249eef75af404c6b9f7741c1e40b9d01 2023-02-12 14:54:05,101 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/49d317637c454f92acb0d83ac9aba3f6 2023-02-12 14:54:05,109 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/49d317637c454f92acb0d83ac9aba3f6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/49d317637c454f92acb0d83ac9aba3f6 2023-02-12 14:54:05,115 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/49d317637c454f92acb0d83ac9aba3f6, entries=1, sequenceid=41, filesize=4.9 K 2023-02-12 14:54:05,116 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 618f7278d7d2801ef6c08ce41da650ea in 851ms, sequenceid=41, compaction requested=true 2023-02-12 14:54:05,116 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:05,116 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:05,116 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:05,116 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:05,116 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,117 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:54:05,117 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:54:05,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,119 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:05,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:05,120 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,120 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:05,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:05,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:05,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:05,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:54:05,121 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@36793177[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:05,121 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:05,121 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,124 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,124 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:05,124 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:54:05,124 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:05,125 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,125 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:05,126 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:05,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:05,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,128 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:54:05,128 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:05,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,128 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:54:05,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:05,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:05,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:05,131 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:54:05,131 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:05,131 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:05,131 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,131 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:05,133 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,133 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,133 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:05,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:05,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:05,133 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:05,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:05,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:05,134 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:05,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:05,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:05,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:05,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithMonthlyPolicy' 2023-02-12 14:54:05,141 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:05,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:05,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:05,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:05,143 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:05,143 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:05,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:05,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:05,243 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:05,245 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:05,246 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithMonthlyPolicy 2023-02-12 14:54:05,248 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:05,248 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=22, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:05,249 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=22, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214245249 2023-02-12 14:54:05,249 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:05,250 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:54:05,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-12 14:54:05,252 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:05,252 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x1d1fc5d6 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:54:05,257 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@588b8730, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:54:05,257 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:05,700 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103d020b7f6569a4883afea0d6a85779dde to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103d020b7f6569a4883afea0d6a85779dde 2023-02-12 14:54:05,701 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:05,715 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:54:05,716 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40832, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:54:05,723 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:54:05,724 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39054, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:54:05,736 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/571fadc0aa834cbfbc384935140eba97 first=Optional[r5] last=Optional[r6] 2023-02-12 14:54:05,736 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/571fadc0aa834cbfbc384935140eba97}] 2023-02-12 14:54:05,741 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/571fadc0aa834cbfbc384935140eba97 for inclusion in 618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:54:05,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-12 14:54:05,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:05,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:05,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/571fadc0aa834cbfbc384935140eba97 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2uglmkrne8dkarp92ocoigiv7trf30l9n8v4btrhfto05032cg31kb8ptifaavod/family1/571fadc0aa834cbfbc384935140eba97 2023-02-12 14:54:05,748 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2uglmkrne8dkarp92ocoigiv7trf30l9n8v4btrhfto05032cg31kb8ptifaavod/family1/571fadc0aa834cbfbc384935140eba97 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_ 2023-02-12 14:54:05,749 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2uglmkrne8dkarp92ocoigiv7trf30l9n8v4btrhfto05032cg31kb8ptifaavod/family1/571fadc0aa834cbfbc384935140eba97 into 618f7278d7d2801ef6c08ce41da650ea/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_ - updating store file list. 2023-02-12 14:54:05,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_ into 618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:54:05,754 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2uglmkrne8dkarp92ocoigiv7trf30l9n8v4btrhfto05032cg31kb8ptifaavod/family1/571fadc0aa834cbfbc384935140eba97 into 618f7278d7d2801ef6c08ce41da650ea/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_) 2023-02-12 14:54:05,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__2uglmkrne8dkarp92ocoigiv7trf30l9n8v4btrhfto05032cg31kb8ptifaavod/family1/571fadc0aa834cbfbc384935140eba97 2023-02-12 14:54:05,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-12 14:54:05,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-12 14:54:05,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 618f7278d7d2801ef6c08ce41da650ea/family1 is initiating minor compaction 2023-02-12 14:54:05,760 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@4e2cf80; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:05,760 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 618f7278d7d2801ef6c08ce41da650ea/family1 in testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:05,760 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/ccb2f71ce87446a2a5139f193725bc7b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6782d10a3ff94510a5c9544fe9181620, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/cff46623305f459a90f2958b8807635e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a318102869704ee08299e85b78a8e775, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/06b4a08528b041e1844a9c0a8ba2db8f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f1054c48bac741f1851c4c53c416adc2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/d8e67045ba8546bea7a44f095b87f56f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/043a2d82369340cd9a08cae04567cc41, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/49d317637c454f92acb0d83ac9aba3f6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp, totalSize=49.7 K 2023-02-12 14:54:05,760 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting ccb2f71ce87446a2a5139f193725bc7b, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 6782d10a3ff94510a5c9544fe9181620, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting cff46623305f459a90f2958b8807635e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting a318102869704ee08299e85b78a8e775, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 06b4a08528b041e1844a9c0a8ba2db8f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting f1054c48bac741f1851c4c53c416adc2, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting d8e67045ba8546bea7a44f095b87f56f, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 043a2d82369340cd9a08cae04567cc41, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 49d317637c454f92acb0d83ac9aba3f6, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-12 14:54:05,761 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-12 14:54:05,763 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:05,764 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:05,766 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010156ce2e1ed11045a8921880aff87a328e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2016010156ce2e1ed11045a8921880aff87a328e 2023-02-12 14:54:05,768 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103367a768999384938bd9847b8c389fe7c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20160103367a768999384938bd9847b8c389fe7c 2023-02-12 14:54:05,768 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-12 14:54:05,768 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-12 14:54:05,772 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:05,772 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:05,830 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 618f7278d7d2801ef6c08ce41da650ea#family1#compaction#22 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:06,251 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123129c89b79b8ae4317874e0ad8c5ebfea8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e2015123129c89b79b8ae4317874e0ad8c5ebfea8 2023-02-12 14:54:06,253 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:06,254 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/c6e05e01e26a4acda6f262dfa0a28679 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/c6e05e01e26a4acda6f262dfa0a28679 2023-02-12 14:54:06,261 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 618f7278d7d2801ef6c08ce41da650ea/family1 of 618f7278d7d2801ef6c08ce41da650ea into c6e05e01e26a4acda6f262dfa0a28679(size=6.6 K), total size for store is 172.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:54:06,261 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:06,261 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676213645760; duration=0sec 2023-02-12 14:54:06,261 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:06,289 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fd48f9649bbd476e8cff3bf0d94a544b first=Optional[r1] last=Optional[r4] 2023-02-12 14:54:06,289 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fd48f9649bbd476e8cff3bf0d94a544b}] 2023-02-12 14:54:06,292 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fd48f9649bbd476e8cff3bf0d94a544b for inclusion in 618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:54:06,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-12 14:54:06,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:06,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:06,297 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/fd48f9649bbd476e8cff3bf0d94a544b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39jd3em3caqsprtojn6epueqa6d2p76bkear0je9r18d0ra5qtj4erd7fs1slgv5/family1/fd48f9649bbd476e8cff3bf0d94a544b 2023-02-12 14:54:06,301 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39jd3em3caqsprtojn6epueqa6d2p76bkear0je9r18d0ra5qtj4erd7fs1slgv5/family1/fd48f9649bbd476e8cff3bf0d94a544b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_ 2023-02-12 14:54:06,302 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39jd3em3caqsprtojn6epueqa6d2p76bkear0je9r18d0ra5qtj4erd7fs1slgv5/family1/fd48f9649bbd476e8cff3bf0d94a544b into 618f7278d7d2801ef6c08ce41da650ea/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_ - updating store file list. 2023-02-12 14:54:06,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_ into 618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:54:06,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39jd3em3caqsprtojn6epueqa6d2p76bkear0je9r18d0ra5qtj4erd7fs1slgv5/family1/fd48f9649bbd476e8cff3bf0d94a544b into 618f7278d7d2801ef6c08ce41da650ea/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_) 2023-02-12 14:54:06,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__39jd3em3caqsprtojn6epueqa6d2p76bkear0je9r18d0ra5qtj4erd7fs1slgv5/family1/fd48f9649bbd476e8cff3bf0d94a544b 2023-02-12 14:54:06,310 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-12 14:54:06,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 182349 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-12 14:54:06,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 618f7278d7d2801ef6c08ce41da650ea/family1 is initiating minor compaction (all files) 2023-02-12 14:54:06,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6c0d3c4f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:06,312 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 618f7278d7d2801ef6c08ce41da650ea/family1 in testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:54:06,312 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a114d7926fe9418c833680829967390e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/c6e05e01e26a4acda6f262dfa0a28679, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp, totalSize=178.1 K 2023-02-12 14:54:06,313 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting a114d7926fe9418c833680829967390e, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840591 2023-02-12 14:54:06,313 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting c6e05e01e26a4acda6f262dfa0a28679, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240591 2023-02-12 14:54:06,314 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965240591 2023-02-12 14:54:06,314 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:06,315 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:06,319 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512014f6d6a4170524b8d8c22f615b54b60cc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512014f6d6a4170524b8d8c22f615b54b60cc 2023-02-12 14:54:06,320 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512053e1efef7cec6424e87c663b776274753 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512053e1efef7cec6424e87c663b776274753 2023-02-12 14:54:06,322 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512289ae5815d5b934771ae4fa5ca25e7a94f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201512289ae5815d5b934771ae4fa5ca25e7a94f 2023-02-12 14:54:06,324 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c4eb8d0f7d0f4d4dad2b0afb87077f10 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151231c4eb8d0f7d0f4d4dad2b0afb87077f10 2023-02-12 14:54:06,324 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-12 14:54:06,324 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-12 14:54:06,325 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:06,325 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:06,336 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 618f7278d7d2801ef6c08ce41da650ea#family1#compaction#23 average throughput is 14.15 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:06,371 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/ccb2f71ce87446a2a5139f193725bc7b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6782d10a3ff94510a5c9544fe9181620, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/cff46623305f459a90f2958b8807635e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a318102869704ee08299e85b78a8e775, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/06b4a08528b041e1844a9c0a8ba2db8f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f1054c48bac741f1851c4c53c416adc2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/d8e67045ba8546bea7a44f095b87f56f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/043a2d82369340cd9a08cae04567cc41, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/49d317637c454f92acb0d83ac9aba3f6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_] to archive 2023-02-12 14:54:06,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:06,381 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/ccb2f71ce87446a2a5139f193725bc7b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/ccb2f71ce87446a2a5139f193725bc7b 2023-02-12 14:54:06,383 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6782d10a3ff94510a5c9544fe9181620 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6782d10a3ff94510a5c9544fe9181620 2023-02-12 14:54:06,388 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/cff46623305f459a90f2958b8807635e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/cff46623305f459a90f2958b8807635e 2023-02-12 14:54:06,390 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a318102869704ee08299e85b78a8e775 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a318102869704ee08299e85b78a8e775 2023-02-12 14:54:06,393 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/06b4a08528b041e1844a9c0a8ba2db8f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/06b4a08528b041e1844a9c0a8ba2db8f 2023-02-12 14:54:06,395 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f1054c48bac741f1851c4c53c416adc2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f1054c48bac741f1851c4c53c416adc2 2023-02-12 14:54:06,399 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/d8e67045ba8546bea7a44f095b87f56f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/d8e67045ba8546bea7a44f095b87f56f 2023-02-12 14:54:06,400 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/043a2d82369340cd9a08cae04567cc41 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/043a2d82369340cd9a08cae04567cc41 2023-02-12 14:54:06,402 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/49d317637c454f92acb0d83ac9aba3f6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/49d317637c454f92acb0d83ac9aba3f6 2023-02-12 14:54:06,405 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f4f4036ae3a141599f1f4f8343f38bf4_SeqId_44_ 2023-02-12 14:54:06,465 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511308fab21fd49024e18ada7258d494b29a4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511308fab21fd49024e18ada7258d494b29a4 2023-02-12 14:54:06,467 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:06,504 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/31847491ade34893bcd2d7f5ae9b9cea first=Optional[r00] last=Optional[r8] 2023-02-12 14:54:06,505 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/31847491ade34893bcd2d7f5ae9b9cea}] 2023-02-12 14:54:06,508 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/31847491ade34893bcd2d7f5ae9b9cea for inclusion in 618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:54:06,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-12 14:54:06,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:06,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:06,512 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/31847491ade34893bcd2d7f5ae9b9cea to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__67na5etfvqn1rs5voqcmidtj2aeecg1vv8hudgv073puak702k2doqvu7mqr0cto/family1/31847491ade34893bcd2d7f5ae9b9cea 2023-02-12 14:54:06,515 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__67na5etfvqn1rs5voqcmidtj2aeecg1vv8hudgv073puak702k2doqvu7mqr0cto/family1/31847491ade34893bcd2d7f5ae9b9cea as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6a4c5232fbaa428b9452e9a9c153911d_SeqId_49_ 2023-02-12 14:54:06,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__67na5etfvqn1rs5voqcmidtj2aeecg1vv8hudgv073puak702k2doqvu7mqr0cto/family1/31847491ade34893bcd2d7f5ae9b9cea into 618f7278d7d2801ef6c08ce41da650ea/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6a4c5232fbaa428b9452e9a9c153911d_SeqId_49_ - updating store file list. 2023-02-12 14:54:06,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6a4c5232fbaa428b9452e9a9c153911d_SeqId_49_ into 618f7278d7d2801ef6c08ce41da650ea/family1 2023-02-12 14:54:06,521 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__67na5etfvqn1rs5voqcmidtj2aeecg1vv8hudgv073puak702k2doqvu7mqr0cto/family1/31847491ade34893bcd2d7f5ae9b9cea into 618f7278d7d2801ef6c08ce41da650ea/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/6a4c5232fbaa428b9452e9a9c153911d_SeqId_49_) 2023-02-12 14:54:06,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithMonthlyPolicy__67na5etfvqn1rs5voqcmidtj2aeecg1vv8hudgv073puak702k2doqvu7mqr0cto/family1/31847491ade34893bcd2d7f5ae9b9cea 2023-02-12 14:54:06,524 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:06,525 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:06,526 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120249eef75af404c6b9f7741c1e40b9d01 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e20151120249eef75af404c6b9f7741c1e40b9d01 2023-02-12 14:54:06,528 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511283fa22787acbd462bbf96731275332263 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511283fa22787acbd462bbf96731275332263 2023-02-12 14:54:06,529 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511303686c35f1e7740c9ae3d13f0699456dd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/f69248d06ee0aebcc88185d282c3e762/family1/d41d8cd98f00b204e9800998ecf8427e201511303686c35f1e7740c9ae3d13f0699456dd 2023-02-12 14:54:06,529 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-12 14:54:06,529 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-12 14:54:06,530 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:06,530 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:06,530 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:54:06,530 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:54:06,530 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithMonthlyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:06,531 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=22, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:06,533 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=22, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 1.2840 sec 2023-02-12 14:54:06,669 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithMonthlyPolicy Thread=267 (was 247) Potentially hanging thread: hconnection-0x4747e79f-metaLookup-shared--pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4747e79f-shared-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44674 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44704 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44814 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4747e79f-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44618 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44718 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44732 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4747e79f-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44896 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44758 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-254133399_15 at /127.0.0.1:44772 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44652 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ResponseProcessor for block BP-1695427604-172.31.14.131-1676213612779:blk_1073741898_1074 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:118) java.io.FilterInputStream.read(FilterInputStream.java:83) java.io.FilterInputStream.read(FilterInputStream.java:83) org.apache.hadoop.hdfs.protocolPB.PBHelperClient.vintPrefixed(PBHelperClient.java:454) org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:213) org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.run(DataStreamer.java:1080) Potentially hanging thread: Thread-479 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x1d1fc5d6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.DelayQueue.poll(DelayQueue.java:259) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:324) org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$$Lambda$45/657284103.run(Unknown Source) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44630 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/4ea1bb5cb9524543826d1c3b3d16290c block BP-1695427604-172.31.14.131-1676213612779:blk_1073741898_1074 java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x1d1fc5d6-EventThread sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:506) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44692 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x6b94224d-shared-pool-23 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: PacketResponder: BP-1695427604-172.31.14.131-1676213612779:blk_1073741898_1074, type=LAST_IN_PIPELINE java.lang.Object.wait(Native Method) java.lang.Object.wait(Object.java:502) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.waitForAckHead(BlockReceiver.java:1327) org.apache.hadoop.hdfs.server.datanode.BlockReceiver$PacketResponder.run(BlockReceiver.java:1399) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: ReadOnlyZKClient-127.0.0.1:61565@0x1d1fc5d6-SendThread(127.0.0.1:61565) sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:345) org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1223) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44800 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44684 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: IPC Parameter Sending Thread #2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4747e79f-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44640 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44588 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4747e79f-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataStreamer for file /user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231df5c84bb019146c69c2259a582f1cc8a_del java.lang.Object.wait(Native Method) org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:678) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:36198 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44788 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4747e79f-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-254133399_15 at /127.0.0.1:44860 [Receiving block BP-1695427604-172.31.14.131-1676213612779:blk_1073741898_1074] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read1(BufferedInputStream.java:286) java.io.BufferedInputStream.read(BufferedInputStream.java:345) java.io.DataInputStream.read(DataInputStream.java:149) org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:209) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doReadFully(PacketReceiver.java:211) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.doRead(PacketReceiver.java:134) org.apache.hadoop.hdfs.protocol.datatransfer.PacketReceiver.receiveNextPacket(PacketReceiver.java:109) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receivePacket(BlockReceiver.java:528) org.apache.hadoop.hdfs.server.datanode.BlockReceiver.receiveBlock(BlockReceiver.java:968) org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:877) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opWriteBlock(Receiver.java:166) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:290) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-254133399_15 at /127.0.0.1:45264 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1153415233_15 at /127.0.0.1:44670 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=632 (was 588) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 271), ProcessCount=168 (was 168), AvailableMemoryMB=4241 (was 4732) 2023-02-12 14:54:06,684 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=267, OpenFileDescriptor=632, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=168, AvailableMemoryMB=4240 2023-02-12 14:54:06,725 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:54:06,725 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-12 14:54:06,725 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x423651f1 to 127.0.0.1:61565 2023-02-12 14:54:06,725 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:54:06,726 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:54:06,726 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x341f9cde to 127.0.0.1:61565 2023-02-12 14:54:06,726 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:54:06,726 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:54:06,728 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1d1fc5d6 to 127.0.0.1:61565 2023-02-12 14:54:06,728 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:54:06,731 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:06,732 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:54:06,733 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithWeeklyPolicy" procId is: 23 2023-02-12 14:54:06,733 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:54:06,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-12 14:54:06,736 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986 2023-02-12 14:54:06,736 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986 empty. 2023-02-12 14:54:06,737 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986 2023-02-12 14:54:06,737 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithWeeklyPolicy regions 2023-02-12 14:54:06,763 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-12 14:54:06,765 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1af670086541a56fea5102c106da6986, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:54:06,835 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-12 14:54:07,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-12 14:54:07,176 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:07,177 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing 1af670086541a56fea5102c106da6986, disabling compactions & flushes 2023-02-12 14:54:07,177 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,177 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,177 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. after waiting 0 ms 2023-02-12 14:54:07,177 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,177 INFO [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,177 DEBUG [RegionOpenAndInit-testMinorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 1af670086541a56fea5102c106da6986: Waiting for close lock at 1676213647177Disabling compacts and flushes for region at 1676213647177Disabling writes for close at 1676213647177Writing region close event to WAL at 1676213647177Closed at 1676213647177 2023-02-12 14:54:07,179 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:54:07,181 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676213647180"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213647180"}]},"ts":"1676213647180"} 2023-02-12 14:54:07,182 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:54:07,183 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:54:07,184 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213647183"}]},"ts":"1676213647183"} 2023-02-12 14:54:07,185 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-12 14:54:07,193 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1af670086541a56fea5102c106da6986, ASSIGN}] 2023-02-12 14:54:07,195 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1af670086541a56fea5102c106da6986, ASSIGN 2023-02-12 14:54:07,197 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=24, ppid=23, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1af670086541a56fea5102c106da6986, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:54:07,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-12 14:54:07,348 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=1af670086541a56fea5102c106da6986, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:07,348 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676213647348"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213647348"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213647348"}]},"ts":"1676213647348"} 2023-02-12 14:54:07,350 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=25, ppid=24, state=RUNNABLE; OpenRegionProcedure 1af670086541a56fea5102c106da6986, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:07,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1af670086541a56fea5102c106da6986, NAME => 'testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:54:07,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithWeeklyPolicy 1af670086541a56fea5102c106da6986 2023-02-12 14:54:07,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:07,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 1af670086541a56fea5102c106da6986 2023-02-12 14:54:07,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 1af670086541a56fea5102c106da6986 2023-02-12 14:54:07,507 INFO [StoreOpener-1af670086541a56fea5102c106da6986-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 1af670086541a56fea5102c106da6986 2023-02-12 14:54:07,509 DEBUG [StoreOpener-1af670086541a56fea5102c106da6986-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1 2023-02-12 14:54:07,509 DEBUG [StoreOpener-1af670086541a56fea5102c106da6986-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1 2023-02-12 14:54:07,509 INFO [StoreOpener-1af670086541a56fea5102c106da6986-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1af670086541a56fea5102c106da6986 columnFamilyName family1 2023-02-12 14:54:07,511 INFO [StoreOpener-1af670086541a56fea5102c106da6986-1] regionserver.HStore(310): Store=1af670086541a56fea5102c106da6986/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:07,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986 2023-02-12 14:54:07,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986 2023-02-12 14:54:07,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 1af670086541a56fea5102c106da6986 2023-02-12 14:54:07,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:54:07,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 1af670086541a56fea5102c106da6986; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66450595, jitterRate=-0.009808972477912903}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:54:07,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 1af670086541a56fea5102c106da6986: Running coprocessor pre-open hook at 1676213647506Writing region info on filesystem at 1676213647506Initializing all the Stores at 1676213647507 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676213647507Cleaning up temporary data from old regions at 1676213647513 (+6 ms)Cleaning up detritus from prior splits at 1676213647513Running coprocessor post-open hooks at 1676213647517 (+4 ms)Region opened successfully at 1676213647517 2023-02-12 14:54:07,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986., pid=25, masterSystemTime=1676213647502 2023-02-12 14:54:07,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,520 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=24 updating hbase:meta row=1af670086541a56fea5102c106da6986, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:07,521 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676213647520"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213647520"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213647520"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213647520"}]},"ts":"1676213647520"} 2023-02-12 14:54:07,524 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=25, resume processing ppid=24 2023-02-12 14:54:07,524 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=25, ppid=24, state=SUCCESS; OpenRegionProcedure 1af670086541a56fea5102c106da6986, server=jenkins-hbase4.apache.org,45319,1676213617001 in 172 msec 2023-02-12 14:54:07,526 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=24, resume processing ppid=23 2023-02-12 14:54:07,526 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=24, ppid=23, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithWeeklyPolicy, region=1af670086541a56fea5102c106da6986, ASSIGN in 331 msec 2023-02-12 14:54:07,527 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:54:07,527 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213647527"}]},"ts":"1676213647527"} 2023-02-12 14:54:07,529 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-12 14:54:07,531 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=23, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:54:07,533 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=23, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithWeeklyPolicy in 806 msec 2023-02-12 14:54:07,597 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/.tmp/family1/4ea1bb5cb9524543826d1c3b3d16290c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/4ea1bb5cb9524543826d1c3b3d16290c 2023-02-12 14:54:07,605 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 618f7278d7d2801ef6c08ce41da650ea/family1 of 618f7278d7d2801ef6c08ce41da650ea into 4ea1bb5cb9524543826d1c3b3d16290c(size=167.4 K), total size for store is 333.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:54:07,605 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 618f7278d7d2801ef6c08ce41da650ea: 2023-02-12 14:54:07,605 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea., storeName=family1, fileCount=3, fileSize=178.1 K (166.0 K, 6.6 K, 5.5 K), priority=2, time=1676213646311; duration=1sec 2023-02-12 14:54:07,605 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:07,651 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a114d7926fe9418c833680829967390e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/c6e05e01e26a4acda6f262dfa0a28679, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_] to archive 2023-02-12 14:54:07,652 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:07,654 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a114d7926fe9418c833680829967390e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/a114d7926fe9418c833680829967390e 2023-02-12 14:54:07,656 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/c6e05e01e26a4acda6f262dfa0a28679 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/c6e05e01e26a4acda6f262dfa0a28679 2023-02-12 14:54:07,657 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/family1/f3b1c6a75e8040f58df5f58fa490eec9_SeqId_47_ 2023-02-12 14:54:07,838 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=23 2023-02-12 14:54:07,839 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithWeeklyPolicy, procId: 23 completed 2023-02-12 14:54:07,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:07,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,869 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:07,869 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:07,869 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:07,869 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:07,869 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,869 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,871 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:07,871 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:07,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:07,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:07,871 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:07,871 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:07,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,871 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:07,872 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:07,872 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:07,872 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:07,875 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:07,875 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:07,875 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:07,875 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:07,876 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-12 14:54:08,121 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:08,297 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130a60912d38a65444391438b0ab7332838 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151130a60912d38a65444391438b0ab7332838 2023-02-12 14:54:08,701 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/28bd04d1b0914dfc93b4e82f99420061 2023-02-12 14:54:08,706 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/28bd04d1b0914dfc93b4e82f99420061 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/28bd04d1b0914dfc93b4e82f99420061 2023-02-12 14:54:08,711 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/28bd04d1b0914dfc93b4e82f99420061, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-12 14:54:08,712 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 836ms, sequenceid=5, compaction requested=false 2023-02-12 14:54:08,713 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:08,713 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:08,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:08,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:08,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:08,713 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:08,714 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,714 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:08,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:08,715 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,715 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:08,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:08,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:08,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:08,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:08,716 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@365fc2a7[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:08,716 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:08,716 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,718 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,718 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:08,718 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:08,718 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:08,718 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,718 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:08,719 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,719 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:08,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:08,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:08,719 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:08,719 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:08,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:08,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:08,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:08,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:08,721 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:08,721 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:08,721 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:08,721 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:08,721 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,721 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:08,724 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,724 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:08,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:08,724 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:08,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:08,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:08,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,726 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:08,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:08,728 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:08,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:08,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:08,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:08,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:08,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:08,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,729 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,729 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,729 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:08,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:08,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:08,729 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:08,829 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:08,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:08,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:08,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:08,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,836 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:08,836 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:08,836 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:08,836 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:08,837 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,837 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:08,838 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:08,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:08,838 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:08,838 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:08,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,839 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:08,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,839 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,839 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:08,839 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:08,839 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:08,839 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:08,839 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:08,840 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:08,840 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:08,840 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:08,840 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:08,840 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-12 14:54:09,254 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283e213c9de0ed445d8a2c67eaedae102e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201511283e213c9de0ed445d8a2c67eaedae102e 2023-02-12 14:54:09,670 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/2612305f33ec4626a92b49efbe4c31c8 2023-02-12 14:54:09,675 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/2612305f33ec4626a92b49efbe4c31c8 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/2612305f33ec4626a92b49efbe4c31c8 2023-02-12 14:54:09,680 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/2612305f33ec4626a92b49efbe4c31c8, entries=1, sequenceid=9, filesize=5.0 K 2023-02-12 14:54:09,681 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 841ms, sequenceid=9, compaction requested=false 2023-02-12 14:54:09,681 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:09,681 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:09,681 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:09,681 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:09,681 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,681 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:09,681 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:09,683 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,683 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:09,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:09,683 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,683 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:09,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:09,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:09,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:09,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:09,685 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50c457aa[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:09,685 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:09,685 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,686 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,686 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,686 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:09,686 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:09,686 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:09,686 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,686 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:09,688 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:09,688 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,688 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:09,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:09,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:09,688 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:09,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:09,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:09,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:09,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:09,690 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:09,690 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:09,690 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:09,690 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:09,690 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,690 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:09,692 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,692 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:09,692 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:09,692 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:09,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:09,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,693 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:09,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,696 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:09,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:09,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:09,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:09,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:09,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:09,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:09,696 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:09,797 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:09,798 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:09,803 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:09,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:09,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,805 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:09,805 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:09,805 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:09,805 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:09,805 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,805 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,807 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,807 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:09,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:09,807 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:09,807 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:09,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:09,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,808 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,809 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:09,809 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:09,809 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:09,809 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:09,809 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:09,810 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:09,810 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:09,810 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:09,810 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:09,810 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:10,237 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512017836b41886c64e13ac9a1320d3ff4333 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512017836b41886c64e13ac9a1320d3ff4333 2023-02-12 14:54:10,645 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/4d0ccae7e62f450ea7f30d4165f8c241 2023-02-12 14:54:10,655 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/4d0ccae7e62f450ea7f30d4165f8c241 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/4d0ccae7e62f450ea7f30d4165f8c241 2023-02-12 14:54:10,665 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/4d0ccae7e62f450ea7f30d4165f8c241, entries=1, sequenceid=13, filesize=4.9 K 2023-02-12 14:54:10,667 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 857ms, sequenceid=13, compaction requested=true 2023-02-12 14:54:10,667 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:10,667 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:10,667 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:10,667 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:10,667 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,667 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:10,667 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:10,670 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,670 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:10,671 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,671 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:10,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:10,673 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60b21df9[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:10,673 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:10,673 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,674 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,675 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:10,675 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:10,675 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:10,676 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:10,676 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:10,676 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:10,680 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,680 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:10,681 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:10,681 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:10,681 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:10,681 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:10,681 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,681 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:10,687 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,687 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,687 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,687 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:10,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:10,688 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:10,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,690 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,694 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,694 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,694 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:10,694 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:10,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:10,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:10,694 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:10,694 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:10,695 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:10,695 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:10,694 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:10,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:10,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:10,695 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,695 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,695 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,695 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,795 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:10,796 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:10,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:10,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:10,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,802 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:10,802 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:10,802 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:10,802 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:10,802 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,803 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,804 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:10,804 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:10,804 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:10,804 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:10,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,804 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:10,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,805 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:10,805 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:10,805 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:10,805 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:10,805 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:10,805 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:10,805 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:10,805 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:10,806 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:10,806 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:11,222 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057014fafe628a4e9e954347599ae917b3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057014fafe628a4e9e954347599ae917b3 2023-02-12 14:54:11,628 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/429f5ff6bfaa49d884c2eb510393d5b5 2023-02-12 14:54:11,633 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/429f5ff6bfaa49d884c2eb510393d5b5 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/429f5ff6bfaa49d884c2eb510393d5b5 2023-02-12 14:54:11,637 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/429f5ff6bfaa49d884c2eb510393d5b5, entries=1, sequenceid=17, filesize=4.9 K 2023-02-12 14:54:11,638 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 832ms, sequenceid=17, compaction requested=true 2023-02-12 14:54:11,639 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:11,639 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:11,639 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:11,639 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:11,639 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,639 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:11,639 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:11,640 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,640 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:11,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:11,641 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,641 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:11,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:11,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:11,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:11,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:11,642 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fbf7a11[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:11,642 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:11,642 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,644 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,644 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:11,644 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:11,644 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:11,644 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,644 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:11,645 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:11,646 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,646 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:11,646 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:11,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:11,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:11,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:11,649 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:11,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:11,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:11,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,649 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:11,651 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:11,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:11,651 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,651 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:11,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:11,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:11,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:11,652 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:11,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,653 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:11,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:11,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:11,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,656 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:11,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:11,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:11,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:11,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:11,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,657 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,657 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,657 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:11,657 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:11,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:11,657 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:11,757 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:11,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:11,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:11,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:11,765 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,765 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:11,765 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:11,765 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:11,765 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:11,765 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,766 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,768 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:11,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:11,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:11,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:11,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:11,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,768 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:11,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,769 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,769 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:11,769 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:11,769 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:11,769 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:11,769 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:11,769 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:11,769 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:11,769 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:11,769 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:11,769 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:12,183 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512289651b36a2bcc44c48b1dca9640636e43 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512289651b36a2bcc44c48b1dca9640636e43 2023-02-12 14:54:12,589 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/1d2c8c41585c494180a326270f830540 2023-02-12 14:54:12,595 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/1d2c8c41585c494180a326270f830540 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/1d2c8c41585c494180a326270f830540 2023-02-12 14:54:12,599 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/1d2c8c41585c494180a326270f830540, entries=1, sequenceid=21, filesize=4.9 K 2023-02-12 14:54:12,600 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 831ms, sequenceid=21, compaction requested=true 2023-02-12 14:54:12,600 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:12,600 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:12,600 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:12,600 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:12,600 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,600 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:12,600 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:12,602 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:12,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:12,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:12,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:12,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:12,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:12,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:12,604 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:12,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57bdc9bb[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:12,604 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,605 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,605 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:12,605 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:12,605 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:12,605 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,605 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:12,606 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,606 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,606 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:12,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:12,606 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:12,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:12,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:12,608 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:12,608 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:12,608 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:12,608 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,608 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:12,611 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,611 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,611 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:12,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:12,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:12,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:12,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:12,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:12,611 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:12,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,613 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:12,615 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:12,615 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:12,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:12,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,616 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:12,616 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:12,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:12,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:12,716 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:12,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:12,721 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:12,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:12,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,723 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:12,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:12,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:12,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:12,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,724 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:12,724 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,724 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:12,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:12,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:12,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:12,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,725 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:12,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,725 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,726 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:12,726 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:12,726 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:12,726 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:12,726 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:12,726 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:12,726 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:12,726 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:12,726 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:12,726 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:13,034 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:54:13,139 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231c4bbc2dd36e04d78ab7d7245d3987049 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231c4bbc2dd36e04d78ab7d7245d3987049 2023-02-12 14:54:13,545 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/0c190b342b654046801e8642e9e5644a 2023-02-12 14:54:13,550 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/0c190b342b654046801e8642e9e5644a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/0c190b342b654046801e8642e9e5644a 2023-02-12 14:54:13,555 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/0c190b342b654046801e8642e9e5644a, entries=1, sequenceid=25, filesize=4.9 K 2023-02-12 14:54:13,556 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 830ms, sequenceid=25, compaction requested=true 2023-02-12 14:54:13,556 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:13,556 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:13,556 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:13,556 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:13,556 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,556 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:13,556 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:13,558 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,558 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:13,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:13,558 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,558 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:13,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:13,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:13,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:13,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:13,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25b6bde1[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:13,559 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:13,559 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,562 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,562 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:13,562 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:13,562 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,562 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:13,562 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:13,566 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:13,566 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,566 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:13,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:13,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:13,566 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:13,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:13,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:13,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:13,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:13,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:13,569 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:13,569 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:13,569 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:13,569 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,569 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:13,570 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,570 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:13,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:13,570 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:13,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:13,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:13,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:13,571 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:13,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,572 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:13,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:13,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:13,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:13,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:13,576 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:13,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:13,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:13,576 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,577 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,677 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:13,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:13,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:13,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:13,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,683 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:13,684 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:13,684 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:13,684 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:13,684 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,684 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,685 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,685 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:13,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:13,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:13,685 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:13,686 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:13,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,686 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:13,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,686 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,687 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:13,687 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:13,687 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:13,687 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:13,687 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:13,687 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:13,687 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:13,687 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:13,687 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:13,687 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:14,101 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601015a068a4482ac40d499caef02bd483de8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601015a068a4482ac40d499caef02bd483de8 2023-02-12 14:54:14,507 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/24873bd5ff934389b3730330055c964d 2023-02-12 14:54:14,513 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/24873bd5ff934389b3730330055c964d as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/24873bd5ff934389b3730330055c964d 2023-02-12 14:54:14,518 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/24873bd5ff934389b3730330055c964d, entries=1, sequenceid=29, filesize=4.9 K 2023-02-12 14:54:14,519 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 832ms, sequenceid=29, compaction requested=true 2023-02-12 14:54:14,520 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:14,520 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:14,520 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:14,521 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,521 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:14,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:14,522 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,522 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:14,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:14,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:14,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:14,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:14,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@559b317b[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:14,523 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:14,523 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,526 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,526 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:14,526 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:14,526 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:14,526 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,526 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:14,528 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:14,528 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,528 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:14,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:14,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:14,528 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:14,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:14,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:14,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:14,530 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:14,530 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:14,531 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:14,531 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:14,531 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,531 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:14,532 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,532 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:14,532 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:14,533 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:14,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:14,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,534 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:14,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:14,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:14,537 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,537 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:14,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:14,537 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,638 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:14,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:14,643 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:14,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:14,645 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,645 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:14,645 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:14,645 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:14,645 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:14,646 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,646 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,648 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,648 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:14,649 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:14,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:14,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,649 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:14,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,649 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,650 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:14,650 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:14,650 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:14,650 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:14,650 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:14,650 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:15,063 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ff6bc9b2713945b7a968bc417f3045a0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ff6bc9b2713945b7a968bc417f3045a0 2023-02-12 14:54:15,469 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/90bf6ac8a5ba4c539d21392f01064825 2023-02-12 14:54:15,480 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/90bf6ac8a5ba4c539d21392f01064825 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/90bf6ac8a5ba4c539d21392f01064825 2023-02-12 14:54:15,489 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/90bf6ac8a5ba4c539d21392f01064825, entries=1, sequenceid=33, filesize=4.9 K 2023-02-12 14:54:15,490 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 840ms, sequenceid=33, compaction requested=true 2023-02-12 14:54:15,490 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:15,490 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:15,490 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:15,490 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:15,491 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,491 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:15,491 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:15,493 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:15,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:15,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:15,495 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3110a62b[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:15,495 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:15,495 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,496 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,496 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,496 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:15,496 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:15,496 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:15,497 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,497 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:15,499 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:15,499 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:15,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:15,503 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:15,503 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:15,503 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:15,503 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:15,503 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,503 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:15,506 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,506 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:15,506 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,506 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,509 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,511 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,513 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,513 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,513 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:15,513 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:15,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:15,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:15,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:15,513 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:15,513 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:15,513 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:15,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:15,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:15,513 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,514 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:15,514 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,514 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:15,514 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,514 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,614 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:15,615 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:15,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:15,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:15,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,622 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:15,622 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:15,622 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:15,622 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:15,622 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,622 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,623 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,623 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:15,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:15,624 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:15,624 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:15,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:15,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,625 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,625 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:15,625 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:15,625 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:15,625 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:15,625 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:15,625 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:15,625 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:15,625 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:15,625 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:15,625 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:16,041 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509074d9b8b11e6a445329ddab5f0f350d9ff to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201509074d9b8b11e6a445329ddab5f0f350d9ff 2023-02-12 14:54:16,451 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/abbd118224e244d19042aeadbbe80dc5 2023-02-12 14:54:16,456 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/abbd118224e244d19042aeadbbe80dc5 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/abbd118224e244d19042aeadbbe80dc5 2023-02-12 14:54:16,461 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/abbd118224e244d19042aeadbbe80dc5, entries=1, sequenceid=37, filesize=4.9 K 2023-02-12 14:54:16,462 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 836ms, sequenceid=37, compaction requested=true 2023-02-12 14:54:16,462 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:16,462 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:16,462 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:16,462 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:16,462 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,462 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:16,462 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:16,463 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,463 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:16,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:16,464 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,464 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:16,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:16,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:16,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:16,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:16,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@85951a3[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:16,465 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:16,465 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,470 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,470 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:16,470 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:16,470 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,470 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:16,470 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:16,472 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,472 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:16,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:16,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:16,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:16,474 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:16,474 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:16,474 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:16,474 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:16,474 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,474 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:16,475 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,475 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:16,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:16,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,475 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:16,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:16,476 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:16,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:16,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:16,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,477 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:16,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:16,480 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:16,480 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:16,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:16,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:16,480 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:16,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:16,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:16,581 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:16,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:16,585 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:16,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:16,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,587 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:16,587 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:16,587 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:16,587 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:16,588 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,588 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,589 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,589 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:16,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:16,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:16,589 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:16,589 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:16,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:16,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,590 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:16,590 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:16,590 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:16,590 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:16,590 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:16,590 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:16,591 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:16,591 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:16,591 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. started... 2023-02-12 14:54:16,591 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1af670086541a56fea5102c106da6986 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:17,003 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112043153d321346444780a09aba026f1707 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015112043153d321346444780a09aba026f1707 2023-02-12 14:54:17,409 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/b544955cf89842c6934271cb43fae27e 2023-02-12 14:54:17,415 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/b544955cf89842c6934271cb43fae27e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/b544955cf89842c6934271cb43fae27e 2023-02-12 14:54:17,421 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/b544955cf89842c6934271cb43fae27e, entries=1, sequenceid=41, filesize=4.9 K 2023-02-12 14:54:17,422 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 1af670086541a56fea5102c106da6986 in 831ms, sequenceid=41, compaction requested=true 2023-02-12 14:54:17,422 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:17,422 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:17,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:17,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:17,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:17,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:17,424 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:17,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:17,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:17,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:17,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:17,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:17,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1890bcfc[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:17,426 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:17,426 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,427 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,428 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:17,428 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:17,428 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:17,428 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,428 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:17,430 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:17,430 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,430 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:17,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,430 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:17,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:17,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:17,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:17,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:17,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:17,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:17,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:17,436 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:17,436 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:17,436 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:17,436 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,436 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:17,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:17,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:17,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:17,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:17,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:17,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:17,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,443 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:17,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,445 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:17,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,447 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,447 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:17,447 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,447 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:17,447 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithWeeklyPolicy' 2023-02-12 14:54:17,448 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:17,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:17,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:17,447 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:17,447 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:17,448 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,448 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,448 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,448 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:17,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:17,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:17,448 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:17,548 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:17,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:17,551 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMinorCompactionWithWeeklyPolicy 2023-02-12 14:54:17,553 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:17,554 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=26, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:17,555 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:17,555 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=26, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214257555 2023-02-12 14:54:17,571 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-12 14:54:17,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 0 del files, 9 selected files, and 0 irrelevant files table 'testMinorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-12 14:54:17,572 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:17,573 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x7b34e372 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:54:17,588 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3378894, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:54:17,588 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,047 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031eb14a1c79654f30a789486d8a911def to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601031eb14a1c79654f30a789486d8a911def 2023-02-12 14:54:18,048 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:18,066 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:54:18,067 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47560, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:54:18,074 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:54:18,076 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41232, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:54:18,085 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d37e40e822c9440ab4e326523b1d97df first=Optional[r3] last=Optional[r6] 2023-02-12 14:54:18,085 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d37e40e822c9440ab4e326523b1d97df}] 2023-02-12 14:54:18,092 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d37e40e822c9440ab4e326523b1d97df for inclusion in 1af670086541a56fea5102c106da6986/family1 2023-02-12 14:54:18,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-12 14:54:18,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:18,095 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:18,096 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d37e40e822c9440ab4e326523b1d97df to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7d1oj7efslbu1o5e85flfoqpf11sb6m9n9oik3oann2d4au3l4e3fifjhcj2tj41/family1/d37e40e822c9440ab4e326523b1d97df 2023-02-12 14:54:18,099 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7d1oj7efslbu1o5e85flfoqpf11sb6m9n9oik3oann2d4au3l4e3fifjhcj2tj41/family1/d37e40e822c9440ab4e326523b1d97df as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_ 2023-02-12 14:54:18,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7d1oj7efslbu1o5e85flfoqpf11sb6m9n9oik3oann2d4au3l4e3fifjhcj2tj41/family1/d37e40e822c9440ab4e326523b1d97df into 1af670086541a56fea5102c106da6986/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_ - updating store file list. 2023-02-12 14:54:18,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_ into 1af670086541a56fea5102c106da6986/family1 2023-02-12 14:54:18,104 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7d1oj7efslbu1o5e85flfoqpf11sb6m9n9oik3oann2d4au3l4e3fifjhcj2tj41/family1/d37e40e822c9440ab4e326523b1d97df into 1af670086541a56fea5102c106da6986/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_) 2023-02-12 14:54:18,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__7d1oj7efslbu1o5e85flfoqpf11sb6m9n9oik3oann2d4au3l4e3fifjhcj2tj41/family1/d37e40e822c9440ab4e326523b1d97df 2023-02-12 14:54:18,104 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-12 14:54:18,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-12 14:54:18,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 1af670086541a56fea5102c106da6986/family1 is initiating minor compaction 2023-02-12 14:54:18,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1c5b3884; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:18,108 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1af670086541a56fea5102c106da6986/family1 in testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:18,108 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/2612305f33ec4626a92b49efbe4c31c8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/4d0ccae7e62f450ea7f30d4165f8c241, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/429f5ff6bfaa49d884c2eb510393d5b5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/1d2c8c41585c494180a326270f830540, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/0c190b342b654046801e8642e9e5644a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/24873bd5ff934389b3730330055c964d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/90bf6ac8a5ba4c539d21392f01064825, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/abbd118224e244d19042aeadbbe80dc5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/b544955cf89842c6934271cb43fae27e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp, totalSize=50.0 K 2023-02-12 14:54:18,108 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 2612305f33ec4626a92b49efbe4c31c8, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:54:18,108 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 4d0ccae7e62f450ea7f30d4165f8c241, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:54:18,108 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 429f5ff6bfaa49d884c2eb510393d5b5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-12 14:54:18,108 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 1d2c8c41585c494180a326270f830540, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-12 14:54:18,109 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 0c190b342b654046801e8642e9e5644a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-12 14:54:18,109 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 24873bd5ff934389b3730330055c964d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-12 14:54:18,109 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 90bf6ac8a5ba4c539d21392f01064825, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-12 14:54:18,109 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting abbd118224e244d19042aeadbbe80dc5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-12 14:54:18,109 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting b544955cf89842c6934271cb43fae27e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-12 14:54:18,109 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-12 14:54:18,110 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:18,111 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:18,113 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512289651b36a2bcc44c48b1dca9640636e43 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512289651b36a2bcc44c48b1dca9640636e43 2023-02-12 14:54:18,114 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231c4bbc2dd36e04d78ab7d7245d3987049 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20151231c4bbc2dd36e04d78ab7d7245d3987049 2023-02-12 14:54:18,115 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601015a068a4482ac40d499caef02bd483de8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201601015a068a4482ac40d499caef02bd483de8 2023-02-12 14:54:18,117 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ff6bc9b2713945b7a968bc417f3045a0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e20160103ff6bc9b2713945b7a968bc417f3045a0 2023-02-12 14:54:18,117 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-12 14:54:18,117 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-12 14:54:18,117 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:18,142 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1af670086541a56fea5102c106da6986#family1#compaction#34 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:18,957 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120597cac857d0b3469693037eb14684a5a6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e2015120597cac857d0b3469693037eb14684a5a6 2023-02-12 14:54:18,957 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/101e70d2a5f449ea9f8d157c7e5502b6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/101e70d2a5f449ea9f8d157c7e5502b6 2023-02-12 14:54:18,958 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:18,964 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 1af670086541a56fea5102c106da6986/family1 of 1af670086541a56fea5102c106da6986 into 101e70d2a5f449ea9f8d157c7e5502b6(size=6.6 K), total size for store is 171.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:54:18,964 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:18,964 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676213658108; duration=0sec 2023-02-12 14:54:18,964 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:18,989 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/07b0e924a1f74642b63351afca0b89eb first=Optional[r1] last=Optional[r2] 2023-02-12 14:54:18,990 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/07b0e924a1f74642b63351afca0b89eb}] 2023-02-12 14:54:19,002 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/07b0e924a1f74642b63351afca0b89eb for inclusion in 1af670086541a56fea5102c106da6986/family1 2023-02-12 14:54:19,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-12 14:54:19,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:19,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:19,009 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/07b0e924a1f74642b63351afca0b89eb to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v64i5ch6668rl3fau55iungf492ailaloff3ouo4sthsne3d2ti994fvkijcbn4n/family1/07b0e924a1f74642b63351afca0b89eb 2023-02-12 14:54:19,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v64i5ch6668rl3fau55iungf492ailaloff3ouo4sthsne3d2ti994fvkijcbn4n/family1/07b0e924a1f74642b63351afca0b89eb as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_ 2023-02-12 14:54:19,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v64i5ch6668rl3fau55iungf492ailaloff3ouo4sthsne3d2ti994fvkijcbn4n/family1/07b0e924a1f74642b63351afca0b89eb into 1af670086541a56fea5102c106da6986/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_ - updating store file list. 2023-02-12 14:54:19,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_ into 1af670086541a56fea5102c106da6986/family1 2023-02-12 14:54:19,019 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v64i5ch6668rl3fau55iungf492ailaloff3ouo4sthsne3d2ti994fvkijcbn4n/family1/07b0e924a1f74642b63351afca0b89eb into 1af670086541a56fea5102c106da6986/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_) 2023-02-12 14:54:19,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithWeeklyPolicy__v64i5ch6668rl3fau55iungf492ailaloff3ouo4sthsne3d2ti994fvkijcbn4n/family1/07b0e924a1f74642b63351afca0b89eb 2023-02-12 14:54:19,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-12 14:54:19,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 181014 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-12 14:54:19,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 1af670086541a56fea5102c106da6986/family1 is initiating minor compaction (all files) 2023-02-12 14:54:19,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6043a4b7; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:19,021 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 1af670086541a56fea5102c106da6986/family1 in testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:54:19,022 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/28bd04d1b0914dfc93b4e82f99420061, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/101e70d2a5f449ea9f8d157c7e5502b6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp, totalSize=176.8 K 2023-02-12 14:54:19,022 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 28bd04d1b0914dfc93b4e82f99420061, keycount=1000, bloomtype=ROW, size=165.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840591 2023-02-12 14:54:19,023 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 101e70d2a5f449ea9f8d157c7e5502b6, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240591 2023-02-12 14:54:19,023 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448965240591 2023-02-12 14:54:19,024 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:19,024 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:19,027 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512017836b41886c64e13ac9a1320d3ff4333 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512017836b41886c64e13ac9a1320d3ff4333 2023-02-12 14:54:19,029 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057014fafe628a4e9e954347599ae917b3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/fde9c8e8f2088680fca8c7a0bcc19fe9/family1/d41d8cd98f00b204e9800998ecf8427e201512057014fafe628a4e9e954347599ae917b3 2023-02-12 14:54:19,029 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithWeeklyPolicy 2 expired mob files are deleted 2023-02-12 14:54:19,029 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-12 14:54:19,030 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMinorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:19,030 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMinorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:19,030 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithWeeklyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:19,031 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=26, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:19,033 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=26, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.4790 sec 2023-02-12 14:54:19,042 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1af670086541a56fea5102c106da6986#family1#compaction#35 average throughput is 17.18 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:19,067 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/2612305f33ec4626a92b49efbe4c31c8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/4d0ccae7e62f450ea7f30d4165f8c241, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/429f5ff6bfaa49d884c2eb510393d5b5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/1d2c8c41585c494180a326270f830540, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/0c190b342b654046801e8642e9e5644a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/24873bd5ff934389b3730330055c964d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/90bf6ac8a5ba4c539d21392f01064825, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/abbd118224e244d19042aeadbbe80dc5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/b544955cf89842c6934271cb43fae27e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_] to archive 2023-02-12 14:54:19,068 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:19,071 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/2612305f33ec4626a92b49efbe4c31c8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/2612305f33ec4626a92b49efbe4c31c8 2023-02-12 14:54:19,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/4d0ccae7e62f450ea7f30d4165f8c241 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/4d0ccae7e62f450ea7f30d4165f8c241 2023-02-12 14:54:19,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/429f5ff6bfaa49d884c2eb510393d5b5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/429f5ff6bfaa49d884c2eb510393d5b5 2023-02-12 14:54:19,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/1d2c8c41585c494180a326270f830540 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/1d2c8c41585c494180a326270f830540 2023-02-12 14:54:19,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/0c190b342b654046801e8642e9e5644a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/0c190b342b654046801e8642e9e5644a 2023-02-12 14:54:19,084 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/24873bd5ff934389b3730330055c964d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/24873bd5ff934389b3730330055c964d 2023-02-12 14:54:19,086 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/90bf6ac8a5ba4c539d21392f01064825 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/90bf6ac8a5ba4c539d21392f01064825 2023-02-12 14:54:19,088 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/abbd118224e244d19042aeadbbe80dc5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/abbd118224e244d19042aeadbbe80dc5 2023-02-12 14:54:19,090 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/b544955cf89842c6934271cb43fae27e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/b544955cf89842c6934271cb43fae27e 2023-02-12 14:54:19,095 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/c099c6e0ab3a462c829ffbeb868eccfd_SeqId_44_ 2023-02-12 14:54:19,154 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithWeeklyPolicy Thread=255 (was 267), OpenFileDescriptor=599 (was 632), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=260 (was 254) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=4034 (was 4240) 2023-02-12 14:54:19,165 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255, OpenFileDescriptor=599, MaxFileDescriptor=60000, SystemLoadAverage=260, ProcessCount=168, AvailableMemoryMB=4034 2023-02-12 14:54:19,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-12 14:54:19,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:19,171 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:54:19,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testScannerOnBulkLoadRefHFiles" procId is: 27 2023-02-12 14:54:19,172 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:54:19,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-12 14:54:19,174 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,174 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,174 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,175 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba empty. 2023-02-12 14:54:19,175 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679 empty. 2023-02-12 14:54:19,175 DEBUG [HFileArchiver-7] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe empty. 2023-02-12 14:54:19,175 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,175 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,175 DEBUG [HFileArchiver-7] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,175 DEBUG [PEWorker-3] procedure.DeleteTableProcedure(328): Archived testScannerOnBulkLoadRefHFiles regions 2023-02-12 14:54:19,183 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testScannerOnBulkLoadRefHFiles/.tabledesc/.tableinfo.0000000001 2023-02-12 14:54:19,186 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(7671): creating {ENCODED => 6a7186591bf78b4981ff4a06040b98fe, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:54:19,186 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(7671): creating {ENCODED => 966e620c05fc39d1026d2c1ff17e6679, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:54:19,186 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(7671): creating {ENCODED => 27e64129bc866c604e8923b6312886ba, NAME => 'testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testScannerOnBulkLoadRefHFiles', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:54:19,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-12 14:54:19,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-12 14:54:19,600 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:19,600 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1603): Closing 966e620c05fc39d1026d2c1ff17e6679, disabling compactions & flushes 2023-02-12 14:54:19,600 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:19,600 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:19,600 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. after waiting 0 ms 2023-02-12 14:54:19,600 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:19,600 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:19,600 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-1] regionserver.HRegion(1557): Region close journal for 966e620c05fc39d1026d2c1ff17e6679: Waiting for close lock at 1676213659600Disabling compacts and flushes for region at 1676213659600Disabling writes for close at 1676213659600Writing region close event to WAL at 1676213659600Closed at 1676213659600 2023-02-12 14:54:19,601 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:19,601 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1603): Closing 6a7186591bf78b4981ff4a06040b98fe, disabling compactions & flushes 2023-02-12 14:54:19,601 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:19,601 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:19,601 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. after waiting 0 ms 2023-02-12 14:54:19,601 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:19,601 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:19,601 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-2] regionserver.HRegion(1557): Region close journal for 6a7186591bf78b4981ff4a06040b98fe: Waiting for close lock at 1676213659601Disabling compacts and flushes for region at 1676213659601Disabling writes for close at 1676213659601Writing region close event to WAL at 1676213659601Closed at 1676213659601 2023-02-12 14:54:19,602 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:19,602 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1603): Closing 27e64129bc866c604e8923b6312886ba, disabling compactions & flushes 2023-02-12 14:54:19,602 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:19,602 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:19,602 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. after waiting 0 ms 2023-02-12 14:54:19,602 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:19,602 INFO [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:19,602 DEBUG [RegionOpenAndInit-testScannerOnBulkLoadRefHFiles-pool-0] regionserver.HRegion(1557): Region close journal for 27e64129bc866c604e8923b6312886ba: Waiting for close lock at 1676213659602Disabling compacts and flushes for region at 1676213659602Disabling writes for close at 1676213659602Writing region close event to WAL at 1676213659602Closed at 1676213659602 2023-02-12 14:54:19,605 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:54:19,606 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676213659605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213659605"}]},"ts":"1676213659605"} 2023-02-12 14:54:19,606 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676213659605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213659605"}]},"ts":"1676213659605"} 2023-02-12 14:54:19,606 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676213659605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213659605"}]},"ts":"1676213659605"} 2023-02-12 14:54:19,608 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-12 14:54:19,609 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:54:19,610 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213659609"}]},"ts":"1676213659609"} 2023-02-12 14:54:19,611 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLING in hbase:meta 2023-02-12 14:54:19,614 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=27e64129bc866c604e8923b6312886ba, ASSIGN}, {pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=966e620c05fc39d1026d2c1ff17e6679, ASSIGN}, {pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6a7186591bf78b4981ff4a06040b98fe, ASSIGN}] 2023-02-12 14:54:19,615 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=966e620c05fc39d1026d2c1ff17e6679, ASSIGN 2023-02-12 14:54:19,615 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=27e64129bc866c604e8923b6312886ba, ASSIGN 2023-02-12 14:54:19,616 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6a7186591bf78b4981ff4a06040b98fe, ASSIGN 2023-02-12 14:54:19,616 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=29, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=966e620c05fc39d1026d2c1ff17e6679, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:54:19,616 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=28, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=27e64129bc866c604e8923b6312886ba, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:54:19,616 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=30, ppid=27, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6a7186591bf78b4981ff4a06040b98fe, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:54:19,768 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=6a7186591bf78b4981ff4a06040b98fe, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:19,768 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=27e64129bc866c604e8923b6312886ba, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:19,768 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=966e620c05fc39d1026d2c1ff17e6679, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:19,768 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676213659768"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213659768"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213659768"}]},"ts":"1676213659768"} 2023-02-12 14:54:19,768 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676213659768"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213659768"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213659768"}]},"ts":"1676213659768"} 2023-02-12 14:54:19,768 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676213659768"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213659768"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213659768"}]},"ts":"1676213659768"} 2023-02-12 14:54:19,770 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=30, state=RUNNABLE; OpenRegionProcedure 6a7186591bf78b4981ff4a06040b98fe, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:19,771 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=32, ppid=28, state=RUNNABLE; OpenRegionProcedure 27e64129bc866c604e8923b6312886ba, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:19,771 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=29, state=RUNNABLE; OpenRegionProcedure 966e620c05fc39d1026d2c1ff17e6679, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:19,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-12 14:54:19,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:19,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 6a7186591bf78b4981ff4a06040b98fe, NAME => 'testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe.', STARTKEY => '2', ENDKEY => ''} 2023-02-12 14:54:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,926 INFO [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,927 DEBUG [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1 2023-02-12 14:54:19,927 DEBUG [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1 2023-02-12 14:54:19,928 INFO [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a7186591bf78b4981ff4a06040b98fe columnFamilyName family1 2023-02-12 14:54:19,929 INFO [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] regionserver.HStore(310): Store=6a7186591bf78b4981ff4a06040b98fe/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:19,930 INFO [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,931 DEBUG [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family2 2023-02-12 14:54:19,931 DEBUG [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family2 2023-02-12 14:54:19,931 INFO [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6a7186591bf78b4981ff4a06040b98fe columnFamilyName family2 2023-02-12 14:54:19,932 INFO [StoreOpener-6a7186591bf78b4981ff4a06040b98fe-1] regionserver.HStore(310): Store=6a7186591bf78b4981ff4a06040b98fe/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:19,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:54:19,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:54:19,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:54:19,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 6a7186591bf78b4981ff4a06040b98fe; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68767952, jitterRate=0.02472233772277832}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:54:19,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 6a7186591bf78b4981ff4a06040b98fe: Running coprocessor pre-open hook at 1676213659925Writing region info on filesystem at 1676213659925Initializing all the Stores at 1676213659926 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213659926Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213659926Cleaning up temporary data from old regions at 1676213659934 (+8 ms)Cleaning up detritus from prior splits at 1676213659934Running coprocessor post-open hooks at 1676213659938 (+4 ms)Region opened successfully at 1676213659938 2023-02-12 14:54:19,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe., pid=31, masterSystemTime=1676213659921 2023-02-12 14:54:19,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:19,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:19,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:19,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 966e620c05fc39d1026d2c1ff17e6679, NAME => 'testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679.', STARTKEY => '1', ENDKEY => '2'} 2023-02-12 14:54:19,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:19,941 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=30 updating hbase:meta row=6a7186591bf78b4981ff4a06040b98fe, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:19,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,941 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676213659941"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213659941"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213659941"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213659941"}]},"ts":"1676213659941"} 2023-02-12 14:54:19,942 INFO [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,944 DEBUG [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679/family1 2023-02-12 14:54:19,944 DEBUG [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679/family1 2023-02-12 14:54:19,944 INFO [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 966e620c05fc39d1026d2c1ff17e6679 columnFamilyName family1 2023-02-12 14:54:19,944 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=30 2023-02-12 14:54:19,944 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=30, state=SUCCESS; OpenRegionProcedure 6a7186591bf78b4981ff4a06040b98fe, server=jenkins-hbase4.apache.org,45319,1676213617001 in 172 msec 2023-02-12 14:54:19,945 INFO [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] regionserver.HStore(310): Store=966e620c05fc39d1026d2c1ff17e6679/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:19,945 INFO [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,946 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=6a7186591bf78b4981ff4a06040b98fe, ASSIGN in 330 msec 2023-02-12 14:54:19,947 DEBUG [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679/family2 2023-02-12 14:54:19,947 DEBUG [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679/family2 2023-02-12 14:54:19,948 INFO [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 966e620c05fc39d1026d2c1ff17e6679 columnFamilyName family2 2023-02-12 14:54:19,949 INFO [StoreOpener-966e620c05fc39d1026d2c1ff17e6679-1] regionserver.HStore(310): Store=966e620c05fc39d1026d2c1ff17e6679/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:19,949 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,950 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,951 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:54:19,953 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:54:19,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:54:19,955 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 966e620c05fc39d1026d2c1ff17e6679; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69970272, jitterRate=0.042638301849365234}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:54:19,955 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 966e620c05fc39d1026d2c1ff17e6679: Running coprocessor pre-open hook at 1676213659941Writing region info on filesystem at 1676213659941Initializing all the Stores at 1676213659942 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213659942Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213659942Cleaning up temporary data from old regions at 1676213659950 (+8 ms)Cleaning up detritus from prior splits at 1676213659951 (+1 ms)Running coprocessor post-open hooks at 1676213659955 (+4 ms)Region opened successfully at 1676213659955 2023-02-12 14:54:19,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679., pid=33, masterSystemTime=1676213659921 2023-02-12 14:54:19,957 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:19,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:19,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:19,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 27e64129bc866c604e8923b6312886ba, NAME => 'testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba.', STARTKEY => '', ENDKEY => '1'} 2023-02-12 14:54:19,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testScannerOnBulkLoadRefHFiles 27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,958 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=966e620c05fc39d1026d2c1ff17e6679, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:19,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:19,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,958 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,958 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676213659958"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213659958"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213659958"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213659958"}]},"ts":"1676213659958"} 2023-02-12 14:54:19,959 INFO [StoreOpener-27e64129bc866c604e8923b6312886ba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,961 DEBUG [StoreOpener-27e64129bc866c604e8923b6312886ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba/family1 2023-02-12 14:54:19,961 DEBUG [StoreOpener-27e64129bc866c604e8923b6312886ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba/family1 2023-02-12 14:54:19,961 INFO [StoreOpener-27e64129bc866c604e8923b6312886ba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27e64129bc866c604e8923b6312886ba columnFamilyName family1 2023-02-12 14:54:19,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=29 2023-02-12 14:54:19,961 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=29, state=SUCCESS; OpenRegionProcedure 966e620c05fc39d1026d2c1ff17e6679, server=jenkins-hbase4.apache.org,45319,1676213617001 in 188 msec 2023-02-12 14:54:19,962 INFO [StoreOpener-27e64129bc866c604e8923b6312886ba-1] regionserver.HStore(310): Store=27e64129bc866c604e8923b6312886ba/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:19,962 INFO [StoreOpener-27e64129bc866c604e8923b6312886ba-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,962 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=966e620c05fc39d1026d2c1ff17e6679, ASSIGN in 347 msec 2023-02-12 14:54:19,963 DEBUG [StoreOpener-27e64129bc866c604e8923b6312886ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba/family2 2023-02-12 14:54:19,963 DEBUG [StoreOpener-27e64129bc866c604e8923b6312886ba-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba/family2 2023-02-12 14:54:19,964 INFO [StoreOpener-27e64129bc866c604e8923b6312886ba-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 27e64129bc866c604e8923b6312886ba columnFamilyName family2 2023-02-12 14:54:19,964 INFO [StoreOpener-27e64129bc866c604e8923b6312886ba-1] regionserver.HStore(310): Store=27e64129bc866c604e8923b6312886ba/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:19,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,965 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testScannerOnBulkLoadRefHFiles descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:54:19,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 27e64129bc866c604e8923b6312886ba 2023-02-12 14:54:19,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:54:19,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 27e64129bc866c604e8923b6312886ba; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62775244, jitterRate=-0.06457597017288208}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:54:19,970 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 27e64129bc866c604e8923b6312886ba: Running coprocessor pre-open hook at 1676213659958Writing region info on filesystem at 1676213659958Initializing all the Stores at 1676213659959 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213659959Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213659959Cleaning up temporary data from old regions at 1676213659966 (+7 ms)Cleaning up detritus from prior splits at 1676213659967 (+1 ms)Running coprocessor post-open hooks at 1676213659970 (+3 ms)Region opened successfully at 1676213659970 2023-02-12 14:54:19,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba., pid=32, masterSystemTime=1676213659921 2023-02-12 14:54:19,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:19,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:19,972 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=28 updating hbase:meta row=27e64129bc866c604e8923b6312886ba, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:19,973 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676213659972"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213659972"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213659972"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213659972"}]},"ts":"1676213659972"} 2023-02-12 14:54:19,975 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-02-12 14:54:19,975 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; OpenRegionProcedure 27e64129bc866c604e8923b6312886ba, server=jenkins-hbase4.apache.org,45319,1676213617001 in 203 msec 2023-02-12 14:54:19,977 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=28, resume processing ppid=27 2023-02-12 14:54:19,977 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=28, ppid=27, state=SUCCESS; TransitRegionStateProcedure table=testScannerOnBulkLoadRefHFiles, region=27e64129bc866c604e8923b6312886ba, ASSIGN in 361 msec 2023-02-12 14:54:19,977 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:54:19,978 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testScannerOnBulkLoadRefHFiles","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213659978"}]},"ts":"1676213659978"} 2023-02-12 14:54:19,979 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testScannerOnBulkLoadRefHFiles, state=ENABLED in hbase:meta 2023-02-12 14:54:19,981 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=27, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:54:19,982 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=27, state=SUCCESS; CreateTableProcedure table=testScannerOnBulkLoadRefHFiles in 812 msec 2023-02-12 14:54:20,123 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testScannerOnBulkLoadRefHFiles' 2023-02-12 14:54:20,264 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/.tmp/family1/3a9a1aafee154d7d851c2587baf7f328 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/3a9a1aafee154d7d851c2587baf7f328 2023-02-12 14:54:20,270 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 1af670086541a56fea5102c106da6986/family1 of 1af670086541a56fea5102c106da6986 into 3a9a1aafee154d7d851c2587baf7f328(size=166.4 K), total size for store is 166.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:54:20,270 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 1af670086541a56fea5102c106da6986: 2023-02-12 14:54:20,270 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986., storeName=family1, fileCount=3, fileSize=176.8 K (165.1 K, 6.6 K, 5.2 K), priority=2, time=1676213659021; duration=1sec 2023-02-12 14:54:20,270 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:20,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=27 2023-02-12 14:54:20,277 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testScannerOnBulkLoadRefHFiles, procId: 27 completed 2023-02-12 14:54:20,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:20,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,285 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-12 14:54:20,285 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:20,285 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-12 14:54:20,285 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-12 14:54:20,285 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,285 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,287 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:20,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:20,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:20,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:20,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:20,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:20,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,288 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:20,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,288 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:20,288 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-12 14:54:20,288 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:20,288 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-12 14:54:20,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:20,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. started... 2023-02-12 14:54:20,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:20,289 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:54:20,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 27e64129bc866c604e8923b6312886ba: 2023-02-12 14:54:20,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:20,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. started... 2023-02-12 14:54:20,290 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. started... 2023-02-12 14:54:20,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:20,289 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:20,291 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 966e620c05fc39d1026d2c1ff17e6679: 2023-02-12 14:54:20,290 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 6a7186591bf78b4981ff4a06040b98fe 2/2 column families, dataSize=48 B heapSize=632 B 2023-02-12 14:54:20,291 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:20,291 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:54:20,291 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:54:20,352 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/28bd04d1b0914dfc93b4e82f99420061, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/101e70d2a5f449ea9f8d157c7e5502b6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_] to archive 2023-02-12 14:54:20,353 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:20,355 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/28bd04d1b0914dfc93b4e82f99420061 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/28bd04d1b0914dfc93b4e82f99420061 2023-02-12 14:54:20,356 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/101e70d2a5f449ea9f8d157c7e5502b6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/101e70d2a5f449ea9f8d157c7e5502b6 2023-02-12 14:54:20,357 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/family1/641c48be51de46e0b74d0f8c6c998fb2_SeqId_47_ 2023-02-12 14:54:20,709 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302126366ec66f4e54fc8b74a97ef0df33401 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302126366ec66f4e54fc8b74a97ef0df33401 2023-02-12 14:54:21,115 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=48, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/.tmp/family1/ccefe5ee897947608251b2ec71fcbb68 2023-02-12 14:54:21,120 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/.tmp/family1/ccefe5ee897947608251b2ec71fcbb68 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/ccefe5ee897947608251b2ec71fcbb68 2023-02-12 14:54:21,124 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/ccefe5ee897947608251b2ec71fcbb68, entries=1, sequenceid=5, filesize=4.9 K 2023-02-12 14:54:21,125 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~48 B/48, heapSize ~360 B/360, currentSize=0 B/0 for 6a7186591bf78b4981ff4a06040b98fe in 835ms, sequenceid=5, compaction requested=false 2023-02-12 14:54:21,126 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 6a7186591bf78b4981ff4a06040b98fe: 2023-02-12 14:54:21,126 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:21,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:54:21,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:54:21,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-12 14:54:21,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-12 14:54:21,128 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,128 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:21,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:21,128 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,128 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:21,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:21,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:21,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-12 14:54:21,129 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@140c567c[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:21,129 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-12 14:54:21,130 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,130 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-12 14:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-12 14:54:21,131 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,131 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:21,131 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:21,132 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-12 14:54:21,132 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,132 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:21,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:21,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:21,132 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-12 14:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:21,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-12 14:54:21,135 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-12 14:54:21,135 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:21,135 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:21,135 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,135 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:21,138 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,138 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:21,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,138 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:21,138 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:21,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,139 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:21,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:21,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-12 14:54:21,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:21,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:21,144 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles because node does not exist (not an error) 2023-02-12 14:54:21,144 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:21,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,144 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:21,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,145 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:21,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:21,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:21,145 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:21,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,245 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:21,246 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:21,250 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:21,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testScannerOnBulkLoadRefHFiles was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:21,252 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,252 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testScannerOnBulkLoadRefHFiles' 2023-02-12 14:54:21,252 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:21,252 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' 2023-02-12 14:54:21,252 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testScannerOnBulkLoadRefHFiles', kicking off acquire phase on members. 2023-02-12 14:54:21,252 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,252 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,254 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:21,254 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:21,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:21,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:21,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:21,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:21,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,255 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,255 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testScannerOnBulkLoadRefHFiles is still present, removing 2023-02-12 14:54:21,255 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:21,255 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testScannerOnBulkLoadRefHFiles' with timeout 60000ms 2023-02-12 14:54:21,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:21,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testScannerOnBulkLoadRefHFiles' starting 'acquire' stage 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:21,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:54:21,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. started... 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. started... 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. started... 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 27e64129bc866c604e8923b6312886ba: 2023-02-12 14:54:21,256 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 6a7186591bf78b4981ff4a06040b98fe 2/2 column families, dataSize=90 B heapSize=752 B 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 966e620c05fc39d1026d2c1ff17e6679: 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:54:21,256 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:54:21,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:54:21,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:54:21,671 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302126718f87708a94602a2c1192fe7c16300 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302126718f87708a94602a2c1192fe7c16300 2023-02-12 14:54:22,078 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=90, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/.tmp/family1/80ae7d19936b4a38b5c2b7e985ae9bcb 2023-02-12 14:54:22,083 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/.tmp/family1/80ae7d19936b4a38b5c2b7e985ae9bcb as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/80ae7d19936b4a38b5c2b7e985ae9bcb 2023-02-12 14:54:22,087 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/80ae7d19936b4a38b5c2b7e985ae9bcb, entries=2, sequenceid=9, filesize=5.0 K 2023-02-12 14:54:22,088 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~90 B/90, heapSize ~480 B/480, currentSize=0 B/0 for 6a7186591bf78b4981ff4a06040b98fe in 832ms, sequenceid=9, compaction requested=false 2023-02-12 14:54:22,088 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 6a7186591bf78b4981ff4a06040b98fe: 2023-02-12 14:54:22,088 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:22,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:54:22,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:54:22,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally acquired 2023-02-12 14:54:22,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testScannerOnBulkLoadRefHFiles) in zk 2023-02-12 14:54:22,091 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,091 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:22,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:22,091 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:22,091 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testScannerOnBulkLoadRefHFiles' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:22,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:22,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:22,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testScannerOnBulkLoadRefHFiles' on coordinator 2023-02-12 14:54:22,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@20198b28[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:22,093 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testScannerOnBulkLoadRefHFiles' starting 'in-barrier' execution. 2023-02-12 14:54:22,093 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,094 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,094 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testScannerOnBulkLoadRefHFiles' received 'reached' from coordinator. 2023-02-12 14:54:22,094 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testScannerOnBulkLoadRefHFiles' locally completed 2023-02-12 14:54:22,094 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testScannerOnBulkLoadRefHFiles' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:22,095 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,095 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:22,096 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,096 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testScannerOnBulkLoadRefHFiles' has notified controller of completion 2023-02-12 14:54:22,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:22,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:22,096 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:22,096 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testScannerOnBulkLoadRefHFiles' completed. 2023-02-12 14:54:22,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:22,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:22,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testScannerOnBulkLoadRefHFiles' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:22,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testScannerOnBulkLoadRefHFiles', counting down latch. Waiting for 0 more 2023-02-12 14:54:22,098 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testScannerOnBulkLoadRefHFiles' execution completed 2023-02-12 14:54:22,098 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:22,098 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:22,098 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,098 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testScannerOnBulkLoadRefHFilesincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:22,100 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,100 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:22,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,100 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:22,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:22,100 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:22,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:22,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,101 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:22,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,104 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:22,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:22,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testScannerOnBulkLoadRefHFiles' 2023-02-12 14:54:22,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,105 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:22,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:22,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testScannerOnBulkLoadRefHFiles'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,205 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:22,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:22,215 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testScannerOnBulkLoadRefHFiles 2023-02-12 14:54:22,219 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-12 14:54:22,220 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=34, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-12 14:54:22,221 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:22,221 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=34, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214262221 2023-02-12 14:54:22,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:54:22,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testScannerOnBulkLoadRefHFiles' and column 'family1' 2023-02-12 14:54:22,222 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-12 14:54:22,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x2e5e3de1 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:54:22,230 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@11926396, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:54:22,230 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230212 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-12 14:54:23,072 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212467703e9b18247d1b658af9010092f1d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c20230212467703e9b18247d1b658af9010092f1d 2023-02-12 14:54:23,073 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-12 14:54:23,086 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:54:23,087 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47568, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:54:23,094 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:54:23,096 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:41238, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:54:23,106 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230212/family1/8104e572a300420a82a32cfeb561a147 first=Optional[k0] last=Optional[k1] 2023-02-12 14:54:23,106 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230212/family1/8104e572a300420a82a32cfeb561a147}] 2023-02-12 14:54:23,112 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230212/family1/8104e572a300420a82a32cfeb561a147 for inclusion in 6a7186591bf78b4981ff4a06040b98fe/family1 2023-02-12 14:54:23,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=k0 last=k1 2023-02-12 14:54:23,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-12 14:54:23,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 6a7186591bf78b4981ff4a06040b98fe: 2023-02-12 14:54:23,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230212/family1/8104e572a300420a82a32cfeb561a147 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testScannerOnBulkLoadRefHFiles__l447757pvifd5lujqhbl52rghrufthtfgtpuk1qo86tv863jgeqh5g8okh1hlfb8/family1/8104e572a300420a82a32cfeb561a147 2023-02-12 14:54:23,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testScannerOnBulkLoadRefHFiles__l447757pvifd5lujqhbl52rghrufthtfgtpuk1qo86tv863jgeqh5g8okh1hlfb8/family1/8104e572a300420a82a32cfeb561a147 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_ 2023-02-12 14:54:23,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testScannerOnBulkLoadRefHFiles__l447757pvifd5lujqhbl52rghrufthtfgtpuk1qo86tv863jgeqh5g8okh1hlfb8/family1/8104e572a300420a82a32cfeb561a147 into 6a7186591bf78b4981ff4a06040b98fe/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_ - updating store file list. 2023-02-12 14:54:23,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_ into 6a7186591bf78b4981ff4a06040b98fe/family1 2023-02-12 14:54:23,121 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testScannerOnBulkLoadRefHFiles__l447757pvifd5lujqhbl52rghrufthtfgtpuk1qo86tv863jgeqh5g8okh1hlfb8/family1/8104e572a300420a82a32cfeb561a147 into 6a7186591bf78b4981ff4a06040b98fe/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_) 2023-02-12 14:54:23,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testScannerOnBulkLoadRefHFiles__l447757pvifd5lujqhbl52rghrufthtfgtpuk1qo86tv863jgeqh5g8okh1hlfb8/family1/8104e572a300420a82a32cfeb561a147 2023-02-12 14:54:23,122 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-12 14:54:23,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 15445 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-12 14:54:23,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 6a7186591bf78b4981ff4a06040b98fe/family1 is initiating minor compaction (all files) 2023-02-12 14:54:23,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@6391b3e4; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:23,123 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 6a7186591bf78b4981ff4a06040b98fe/family1 in testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:54:23,123 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/ccefe5ee897947608251b2ec71fcbb68, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/80ae7d19936b4a38b5c2b7e985ae9bcb, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/.tmp, totalSize=15.1 K 2023-02-12 14:54:23,124 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting ccefe5ee897947608251b2ec71fcbb68, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213660277 2023-02-12 14:54:23,124 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 80ae7d19936b4a38b5c2b7e985ae9bcb, keycount=2, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213660277 2023-02-12 14:54:23,125 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213660277 2023-02-12 14:54:23,125 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-12 14:54:23,126 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:23,128 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302126366ec66f4e54fc8b74a97ef0df33401 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302126366ec66f4e54fc8b74a97ef0df33401 2023-02-12 14:54:23,129 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302126718f87708a94602a2c1192fe7c16300 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testScannerOnBulkLoadRefHFiles/9a980ee3ec24e1d8bfa59c2e531fd315/family1/c81e728d9d4c2f636f067f89cc14862c202302126718f87708a94602a2c1192fe7c16300 2023-02-12 14:54:23,129 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testScannerOnBulkLoadRefHFiles 2 expired mob files are deleted 2023-02-12 14:54:23,129 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testScannerOnBulkLoadRefHFiles/c81e728d9d4c2f636f067f89cc14862c20230212' 2023-02-12 14:54:23,129 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230212 for table='testScannerOnBulkLoadRefHFiles' and column='family1' 2023-02-12 14:54:23,129 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testScannerOnBulkLoadRefHFiles' column='family1' 2023-02-12 14:54:23,130 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testScannerOnBulkLoadRefHFiles', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:23,131 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=34, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE 2023-02-12 14:54:23,132 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=34, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testScannerOnBulkLoadRefHFiles.mobLock, type=EXCLUSIVE in 914 msec 2023-02-12 14:54:23,135 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6a7186591bf78b4981ff4a06040b98fe#family1#compaction#38 average throughput is 0.06 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:23,162 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testScannerOnBulkLoadRefHFiles Thread=255 (was 255), OpenFileDescriptor=606 (was 599) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 260), ProcessCount=168 (was 168), AvailableMemoryMB=4023 (was 4034) 2023-02-12 14:54:23,173 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=255, OpenFileDescriptor=606, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=168, AvailableMemoryMB=4023 2023-02-12 14:54:23,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-12 14:54:23,175 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:23,176 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:54:23,177 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyPolicy" procId is: 35 2023-02-12 14:54:23,177 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:54:23,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 14:54:23,178 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:23,179 DEBUG [HFileArchiver-8] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006 empty. 2023-02-12 14:54:23,179 DEBUG [HFileArchiver-8] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:23,179 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyPolicy regions 2023-02-12 14:54:23,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 14:54:23,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 14:54:23,587 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-12 14:54:23,588 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => ac5d58614983383110f00b3bb838c006, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:54:23,780 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 14:54:23,994 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:23,994 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1603): Closing ac5d58614983383110f00b3bb838c006, disabling compactions & flushes 2023-02-12 14:54:23,994 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:23,994 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:23,994 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. after waiting 0 ms 2023-02-12 14:54:23,994 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:23,994 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:23,994 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for ac5d58614983383110f00b3bb838c006: Waiting for close lock at 1676213663994Disabling compacts and flushes for region at 1676213663994Disabling writes for close at 1676213663994Writing region close event to WAL at 1676213663994Closed at 1676213663994 2023-02-12 14:54:23,996 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:54:23,997 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676213663997"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213663997"}]},"ts":"1676213663997"} 2023-02-12 14:54:23,999 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:54:23,999 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:54:24,000 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213664000"}]},"ts":"1676213664000"} 2023-02-12 14:54:24,001 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLING in hbase:meta 2023-02-12 14:54:24,003 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=ac5d58614983383110f00b3bb838c006, ASSIGN}] 2023-02-12 14:54:24,005 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=ac5d58614983383110f00b3bb838c006, ASSIGN 2023-02-12 14:54:24,005 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=ac5d58614983383110f00b3bb838c006, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:54:24,156 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=ac5d58614983383110f00b3bb838c006, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:24,157 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676213664156"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213664156"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213664156"}]},"ts":"1676213664156"} 2023-02-12 14:54:24,158 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=37, ppid=36, state=RUNNABLE; OpenRegionProcedure ac5d58614983383110f00b3bb838c006, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:24,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 14:54:24,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:24,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => ac5d58614983383110f00b3bb838c006, NAME => 'testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:54:24,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyPolicy ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:24,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:24,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:24,314 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:24,315 INFO [StoreOpener-ac5d58614983383110f00b3bb838c006-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:24,318 DEBUG [StoreOpener-ac5d58614983383110f00b3bb838c006-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1 2023-02-12 14:54:24,318 DEBUG [StoreOpener-ac5d58614983383110f00b3bb838c006-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1 2023-02-12 14:54:24,318 INFO [StoreOpener-ac5d58614983383110f00b3bb838c006-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ac5d58614983383110f00b3bb838c006 columnFamilyName family1 2023-02-12 14:54:24,320 INFO [StoreOpener-ac5d58614983383110f00b3bb838c006-1] regionserver.HStore(310): Store=ac5d58614983383110f00b3bb838c006/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:24,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:24,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:24,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for ac5d58614983383110f00b3bb838c006 2023-02-12 14:54:24,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:54:24,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened ac5d58614983383110f00b3bb838c006; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67991102, jitterRate=0.013146370649337769}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:54:24,325 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for ac5d58614983383110f00b3bb838c006: Running coprocessor pre-open hook at 1676213664314Writing region info on filesystem at 1676213664314Initializing all the Stores at 1676213664314Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676213664315 (+1 ms)Cleaning up temporary data from old regions at 1676213664321 (+6 ms)Cleaning up detritus from prior splits at 1676213664322 (+1 ms)Running coprocessor post-open hooks at 1676213664325 (+3 ms)Region opened successfully at 1676213664325 2023-02-12 14:54:24,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006., pid=37, masterSystemTime=1676213664310 2023-02-12 14:54:24,327 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:24,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:24,328 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=ac5d58614983383110f00b3bb838c006, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:24,328 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676213664328"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213664328"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213664328"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213664328"}]},"ts":"1676213664328"} 2023-02-12 14:54:24,331 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=36 2023-02-12 14:54:24,331 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=36, state=SUCCESS; OpenRegionProcedure ac5d58614983383110f00b3bb838c006, server=jenkins-hbase4.apache.org,45319,1676213617001 in 171 msec 2023-02-12 14:54:24,332 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-12 14:54:24,332 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyPolicy, region=ac5d58614983383110f00b3bb838c006, ASSIGN in 328 msec 2023-02-12 14:54:24,333 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:54:24,333 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213664333"}]},"ts":"1676213664333"} 2023-02-12 14:54:24,334 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyPolicy, state=ENABLED in hbase:meta 2023-02-12 14:54:24,337 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=35, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:54:24,338 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyPolicy in 1.1630 sec 2023-02-12 14:54:24,353 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/.tmp/family1/2d527488c4004420b0bf85e8e729e172 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/2d527488c4004420b0bf85e8e729e172 2023-02-12 14:54:24,357 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 6a7186591bf78b4981ff4a06040b98fe/family1 of 6a7186591bf78b4981ff4a06040b98fe into 2d527488c4004420b0bf85e8e729e172(size=5.1 K), total size for store is 5.1 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:54:24,358 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 6a7186591bf78b4981ff4a06040b98fe: 2023-02-12 14:54:24,358 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe., storeName=family1, fileCount=3, fileSize=15.1 K (4.9 K, 5.0 K, 5.2 K), priority=2, time=1676213663123; duration=1sec 2023-02-12 14:54:24,358 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:24,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/ccefe5ee897947608251b2ec71fcbb68, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/80ae7d19936b4a38b5c2b7e985ae9bcb, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_] to archive 2023-02-12 14:54:24,454 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:24,455 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/ccefe5ee897947608251b2ec71fcbb68 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/ccefe5ee897947608251b2ec71fcbb68 2023-02-12 14:54:24,457 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/80ae7d19936b4a38b5c2b7e985ae9bcb to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/80ae7d19936b4a38b5c2b7e985ae9bcb 2023-02-12 14:54:24,458 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/family1/d62b148754d04b8d8af1b3c0c12e69b3_SeqId_12_ 2023-02-12 14:54:25,282 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-12 14:54:25,282 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyPolicy, procId: 35 completed 2023-02-12 14:54:25,310 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:25,312 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,312 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:25,312 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:25,312 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:25,312 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:25,313 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,313 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,314 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:25,314 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:25,314 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:25,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:25,315 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:25,315 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:25,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,315 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:25,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,316 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,316 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:25,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:25,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:25,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:25,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:25,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:25,316 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:25,316 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:25,317 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-12 14:54:25,450 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:54:25,748 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130b549fad94a524b87b8d4907bfe190182 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b549fad94a524b87b8d4907bfe190182 2023-02-12 14:54:26,124 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:26,152 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/7c82fa3d8fbc4d51abb736b61d02d97c 2023-02-12 14:54:26,157 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/7c82fa3d8fbc4d51abb736b61d02d97c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/7c82fa3d8fbc4d51abb736b61d02d97c 2023-02-12 14:54:26,161 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/7c82fa3d8fbc4d51abb736b61d02d97c, entries=1000, sequenceid=5, filesize=165.1 K 2023-02-12 14:54:26,162 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 845ms, sequenceid=5, compaction requested=false 2023-02-12 14:54:26,163 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:26,163 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:26,163 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:26,163 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:26,163 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,163 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:26,163 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:26,165 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,165 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:26,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:26,165 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,165 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:26,167 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@620f12c1[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:26,167 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:26,167 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,168 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,168 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:26,168 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:26,168 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:26,168 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,168 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:26,171 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:26,171 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,171 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:26,172 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:26,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:26,174 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:26,174 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:26,174 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:26,174 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,174 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:26,175 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,175 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:26,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,175 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:26,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:26,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:26,176 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,177 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:26,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,179 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:26,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:26,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,179 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:26,179 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:26,280 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:26,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:26,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:26,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:26,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,286 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:26,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:26,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:26,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:26,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,287 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,288 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:26,288 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:26,289 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:26,289 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,289 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,290 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,290 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:26,290 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:26,290 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:26,290 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:26,290 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:26,290 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:26,290 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:26,290 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:26,290 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:26,290 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-12 14:54:26,705 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112883e64e75a4ec48838ee2f4e4636086bf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015112883e64e75a4ec48838ee2f4e4636086bf 2023-02-12 14:54:27,112 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/24878ff5f31e4f1ba833c05f6c3e9b79 2023-02-12 14:54:27,117 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/24878ff5f31e4f1ba833c05f6c3e9b79 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/24878ff5f31e4f1ba833c05f6c3e9b79 2023-02-12 14:54:27,120 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/24878ff5f31e4f1ba833c05f6c3e9b79, entries=1, sequenceid=9, filesize=5.0 K 2023-02-12 14:54:27,121 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 831ms, sequenceid=9, compaction requested=false 2023-02-12 14:54:27,121 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:27,121 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:27,121 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:27,121 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:27,121 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,122 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:27,122 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:27,129 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:27,131 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f652520[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:27,131 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:27,131 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,135 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:27,135 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:27,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:27,137 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:27,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:27,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:27,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:27,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:27,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:27,140 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:27,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:27,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:27,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,140 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:27,143 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,143 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,143 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:27,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:27,143 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,145 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:27,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:27,148 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:27,148 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:27,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:27,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:27,148 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:27,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:27,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:27,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:27,249 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:27,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:27,253 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:27,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:27,255 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,255 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:27,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:27,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:27,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:27,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,257 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,257 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:27,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:27,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:27,257 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:27,257 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:27,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,257 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:27,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,258 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,258 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:27,258 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:27,258 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:27,258 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:27,258 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:27,262 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:27,262 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:27,262 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:27,263 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:27,263 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:27,675 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120170fc5f0b05814968ba499b8a3874c472 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120170fc5f0b05814968ba499b8a3874c472 2023-02-12 14:54:28,081 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/605b3df86de14db4b855b7e71bd4f1f7 2023-02-12 14:54:28,087 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/605b3df86de14db4b855b7e71bd4f1f7 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/605b3df86de14db4b855b7e71bd4f1f7 2023-02-12 14:54:28,092 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/605b3df86de14db4b855b7e71bd4f1f7, entries=1, sequenceid=13, filesize=4.9 K 2023-02-12 14:54:28,093 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 830ms, sequenceid=13, compaction requested=true 2023-02-12 14:54:28,093 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:28,093 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:28,093 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:28,093 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:28,093 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,093 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:28,093 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:28,094 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,094 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:28,095 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,095 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:28,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:28,097 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9ac3306[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:28,097 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:28,097 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,099 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:28,099 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:28,099 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,099 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:28,099 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:28,100 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,100 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:28,101 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:28,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:28,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:28,103 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:28,103 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:28,103 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:28,103 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:28,103 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,103 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:28,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:28,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:28,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:28,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:28,106 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:28,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,107 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,109 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:28,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:28,110 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:28,110 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:28,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:28,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:28,210 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:28,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:28,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:28,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:28,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,216 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:28,216 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:28,216 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:28,216 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:28,217 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,217 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,219 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:28,219 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:28,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:28,220 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:28,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,220 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:28,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:28,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:28,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:28,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:28,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:28,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:28,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:28,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:28,221 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:28,221 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:28,221 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:28,635 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057b5c7476b61845d4a82c4bd6312c250d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512057b5c7476b61845d4a82c4bd6312c250d 2023-02-12 14:54:29,049 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/9fe9f6a64a864d87a1b1ad750b21e6a0 2023-02-12 14:54:29,054 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/9fe9f6a64a864d87a1b1ad750b21e6a0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/9fe9f6a64a864d87a1b1ad750b21e6a0 2023-02-12 14:54:29,060 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/9fe9f6a64a864d87a1b1ad750b21e6a0, entries=1, sequenceid=17, filesize=4.9 K 2023-02-12 14:54:29,061 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 840ms, sequenceid=17, compaction requested=true 2023-02-12 14:54:29,061 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:29,061 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:29,061 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:29,061 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:29,061 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,061 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:29,061 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:29,063 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,063 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:29,063 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,063 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:29,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:29,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:29,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6500a9d0[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:29,065 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:29,065 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,066 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,066 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:29,066 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:29,066 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:29,066 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,066 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:29,068 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:29,068 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:29,068 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,068 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:29,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:29,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:29,071 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:29,071 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:29,071 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:29,071 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:29,071 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,071 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:29,073 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,073 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:29,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,073 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:29,073 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:29,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,074 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:29,077 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:29,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:29,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:29,077 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:29,077 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:29,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:29,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:29,177 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:29,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:29,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:29,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:29,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,184 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:29,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:29,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:29,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:29,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,185 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,185 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:29,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:29,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:29,185 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:29,185 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:29,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,186 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:29,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,186 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,186 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:29,186 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:29,186 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:29,186 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:29,186 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:29,187 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:29,187 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:29,187 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:29,187 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:29,187 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:29,600 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512284803a4b6bfc34c668875a5c6c48a16cb to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284803a4b6bfc34c668875a5c6c48a16cb 2023-02-12 14:54:30,007 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/25ecd6f3887544e4b66122c7a29aa23a 2023-02-12 14:54:30,013 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/25ecd6f3887544e4b66122c7a29aa23a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/25ecd6f3887544e4b66122c7a29aa23a 2023-02-12 14:54:30,018 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/25ecd6f3887544e4b66122c7a29aa23a, entries=1, sequenceid=21, filesize=4.9 K 2023-02-12 14:54:30,019 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 832ms, sequenceid=21, compaction requested=true 2023-02-12 14:54:30,019 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:30,019 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:30,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:30,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:30,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:30,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:30,023 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,023 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:30,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:30,023 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,024 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:30,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:30,025 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7f0326ab[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:30,025 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:30,025 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,027 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,028 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,028 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:30,028 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:30,028 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:30,028 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,028 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:30,029 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:30,029 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,029 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:30,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:30,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:30,029 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:30,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:30,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:30,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:30,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:30,032 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:30,032 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:30,032 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:30,032 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:30,032 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,032 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:30,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,035 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:30,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:30,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:30,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,035 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:30,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:30,038 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:30,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:30,041 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:30,041 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:30,041 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:30,041 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:30,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:30,042 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:30,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:30,142 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:30,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:30,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:30,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:30,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,148 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:30,148 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:30,148 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:30,148 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:30,149 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,149 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,151 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,151 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:30,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:30,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:30,152 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,152 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:30,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,152 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:30,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,153 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,155 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:30,155 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,155 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:30,155 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:30,155 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:30,155 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:30,155 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:30,155 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:30,155 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:30,156 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:30,178 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512316f3f7c4255d8457bb6ae7325fed3f539 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512316f3f7c4255d8457bb6ae7325fed3f539 2023-02-12 14:54:30,584 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/ab6d13d6b57a452bbdc4a3645ac17c04 2023-02-12 14:54:30,590 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/ab6d13d6b57a452bbdc4a3645ac17c04 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/ab6d13d6b57a452bbdc4a3645ac17c04 2023-02-12 14:54:30,595 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/ab6d13d6b57a452bbdc4a3645ac17c04, entries=1, sequenceid=25, filesize=4.9 K 2023-02-12 14:54:30,596 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 440ms, sequenceid=25, compaction requested=true 2023-02-12 14:54:30,596 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:30,596 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:30,596 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:30,596 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:30,596 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,596 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:30,597 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:30,598 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,598 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:30,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:30,599 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,599 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:30,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:30,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:30,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:30,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:30,600 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2999504e[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:30,601 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:30,601 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,602 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:30,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:30,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:30,602 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,602 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:30,603 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:30,603 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,603 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:30,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:30,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:30,604 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:30,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:30,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:30,607 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:30,607 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:30,607 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:30,607 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:30,607 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,607 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:30,610 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,610 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:30,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:30,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,610 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:30,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:30,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:30,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:30,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,611 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:30,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,613 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:30,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:30,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:30,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:30,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:30,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,616 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:30,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:30,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:30,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:30,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:30,617 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:30,617 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:30,617 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:30,717 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:30,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:30,723 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:30,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:30,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,724 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:30,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:30,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:30,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:30,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,725 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,726 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:30,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:30,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:30,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:30,727 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:30,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,728 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,728 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:30,728 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:30,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:30,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:30,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:30,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:30,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:30,728 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:30,728 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:30,729 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:30,752 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160101c0ab2fe18ff84cfe9fdd26f5c7798abf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c0ab2fe18ff84cfe9fdd26f5c7798abf 2023-02-12 14:54:31,163 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/57090046f105441ebdd37b22a7a992a1 2023-02-12 14:54:31,168 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/57090046f105441ebdd37b22a7a992a1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/57090046f105441ebdd37b22a7a992a1 2023-02-12 14:54:31,172 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/57090046f105441ebdd37b22a7a992a1, entries=1, sequenceid=29, filesize=4.9 K 2023-02-12 14:54:31,173 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 444ms, sequenceid=29, compaction requested=true 2023-02-12 14:54:31,173 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:31,173 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:31,173 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:31,173 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:31,173 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,173 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:31,173 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:31,175 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:31,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:31,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41456278[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:31,176 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:31,176 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,178 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,179 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:31,179 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,179 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:31,179 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:31,179 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:31,180 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,180 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:31,180 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:31,180 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:31,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:31,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:31,182 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:31,182 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:31,182 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:31,182 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:31,182 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,182 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:31,184 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,184 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:31,184 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:31,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:31,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:31,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:31,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,185 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:31,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:31,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:31,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:31,187 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:31,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:31,187 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:31,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:31,188 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,188 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,188 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,188 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:31,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:31,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:31,188 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,188 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:31,288 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:31,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:31,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:31,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:31,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,295 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:31,295 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:31,295 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:31,295 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:31,295 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,295 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,306 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,306 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:31,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:31,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:31,306 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:31,306 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:31,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,306 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:31,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,307 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:31,307 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:31,307 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:31,307 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:31,307 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:31,307 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:31,307 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:31,307 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:31,307 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:31,307 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:31,720 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601039f789372314643ffba6867d2420c23cf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601039f789372314643ffba6867d2420c23cf 2023-02-12 14:54:32,125 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/b2219a5be7674b9a802a575cc70f33a8 2023-02-12 14:54:32,131 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/b2219a5be7674b9a802a575cc70f33a8 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/b2219a5be7674b9a802a575cc70f33a8 2023-02-12 14:54:32,135 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/b2219a5be7674b9a802a575cc70f33a8, entries=1, sequenceid=33, filesize=4.9 K 2023-02-12 14:54:32,136 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 829ms, sequenceid=33, compaction requested=true 2023-02-12 14:54:32,137 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:32,137 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:32,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:32,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:32,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:32,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:32,138 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:32,139 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,139 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:32,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:32,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@216b1da9[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:32,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:32,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,142 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:32,142 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:32,142 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:32,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:32,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:32,143 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:32,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:32,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:32,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:32,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:32,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:32,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:32,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:32,146 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:32,146 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:32,146 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:32,146 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:32,146 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,146 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:32,149 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,149 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:32,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,149 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:32,149 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:32,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:32,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,150 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,152 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,152 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:32,152 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:32,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:32,153 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:32,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:32,153 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:32,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:32,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,153 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,253 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:32,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:32,258 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:32,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:32,260 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,260 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:32,260 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:32,260 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:32,260 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:32,260 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,260 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,261 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,262 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:32,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:32,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:32,262 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:32,262 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:32,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,262 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:32,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,263 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,263 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:32,263 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:32,263 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:32,263 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:32,263 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:32,263 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:32,263 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:32,263 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:32,263 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:32,263 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:32,676 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907fe84d0b3b5ad4348ba40170e16fbf53d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20150907fe84d0b3b5ad4348ba40170e16fbf53d 2023-02-12 14:54:33,082 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/60cd5061548948f1999a2940be5bcd9e 2023-02-12 14:54:33,087 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/60cd5061548948f1999a2940be5bcd9e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/60cd5061548948f1999a2940be5bcd9e 2023-02-12 14:54:33,091 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/60cd5061548948f1999a2940be5bcd9e, entries=1, sequenceid=37, filesize=4.9 K 2023-02-12 14:54:33,091 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 828ms, sequenceid=37, compaction requested=true 2023-02-12 14:54:33,092 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:33,092 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:33,092 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:33,092 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:33,092 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,092 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:33,092 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:33,093 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,093 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:33,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:33,094 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,094 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:33,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:33,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:33,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:33,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:33,095 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47bb4151[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:33,095 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:33,095 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,097 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,097 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,097 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:33,097 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:33,097 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:33,097 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,097 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:33,098 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,098 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:33,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:33,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:33,098 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:33,098 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:33,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:33,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:33,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:33,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:33,100 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:33,100 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:33,101 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:33,101 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:33,101 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,101 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:33,102 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,102 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:33,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:33,102 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:33,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:33,102 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:33,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:33,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:33,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,103 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:33,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,105 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:33,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:33,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:33,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:33,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:33,106 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:33,105 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:33,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:33,106 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,106 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,106 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:33,106 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:33,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:33,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:33,206 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:33,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:33,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:33,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:33,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,212 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:33,212 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:33,213 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' 2023-02-12 14:54:33,213 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyPolicy', kicking off acquire phase on members. 2023-02-12 14:54:33,213 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,213 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,215 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,215 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:33,216 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:33,216 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:33,216 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,216 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:33,216 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,216 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,217 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyPolicy is still present, removing 2023-02-12 14:54:33,217 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:33,217 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyPolicy' with timeout 60000ms 2023-02-12 14:54:33,217 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:33,217 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyPolicy' starting 'acquire' stage 2023-02-12 14:54:33,217 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:33,217 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:33,217 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:33,217 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. started... 2023-02-12 14:54:33,217 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing ac5d58614983383110f00b3bb838c006 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:33,630 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120d0ff78c470474a38a3be1be634b7a9f5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151120d0ff78c470474a38a3be1be634b7a9f5 2023-02-12 14:54:34,036 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/553a6ebcd73b4167b0733d055ee2c60b 2023-02-12 14:54:34,041 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/553a6ebcd73b4167b0733d055ee2c60b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/553a6ebcd73b4167b0733d055ee2c60b 2023-02-12 14:54:34,045 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/553a6ebcd73b4167b0733d055ee2c60b, entries=1, sequenceid=41, filesize=4.9 K 2023-02-12 14:54:34,046 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for ac5d58614983383110f00b3bb838c006 in 829ms, sequenceid=41, compaction requested=true 2023-02-12 14:54:34,046 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:34,046 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:34,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:34,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:34,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally acquired 2023-02-12 14:54:34,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyPolicy) in zk 2023-02-12 14:54:34,048 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,048 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:34,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:34,048 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,048 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:34,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:34,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:34,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:34,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyPolicy' on coordinator 2023-02-12 14:54:34,049 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyPolicy' starting 'in-barrier' execution. 2023-02-12 14:54:34,049 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@244372c8[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:34,049 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,051 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,051 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyPolicy' received 'reached' from coordinator. 2023-02-12 14:54:34,051 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyPolicy' locally completed 2023-02-12 14:54:34,051 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,051 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:34,051 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:34,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyPolicy' has notified controller of completion 2023-02-12 14:54:34,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:34,052 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyPolicy' completed. 2023-02-12 14:54:34,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:34,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:34,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:34,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:34,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:34,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:34,054 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:54:34,054 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyPolicy' execution completed 2023-02-12 14:54:34,054 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:34,054 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:34,054 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,054 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:34,057 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,057 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:34,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:34,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,057 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:34,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:34,058 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:34,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:34,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:34,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,059 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:34,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,063 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,063 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:34,063 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:34,063 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyPolicy' 2023-02-12 14:54:34,064 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:34,063 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,063 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:34,063 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:34,064 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:34,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy because node does not exist (not an error) 2023-02-12 14:54:34,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:34,064 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:34,064 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:34,064 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,064 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,064 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:34,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:34,164 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:34,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:34,166 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyPolicy 2023-02-12 14:54:34,168 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:34,169 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=38, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:34,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:34,170 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=38, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214274170 2023-02-12 14:54:34,173 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:54:34,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyPolicy' and column 'family1' 2023-02-12 14:54:34,174 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:34,174 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x3447f765 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:54:34,179 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21dd7297, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:54:34,179 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,622 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601035965c7dda2744558a0a8772eb895f222 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601035965c7dda2744558a0a8772eb895f222 2023-02-12 14:54:34,623 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:34,636 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:54:34,637 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:43580, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:54:34,643 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:54:34,644 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:52958, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:54:34,652 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0596ef715668496692c54135f93f9433 first=Optional[r3] last=Optional[r6] 2023-02-12 14:54:34,652 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0596ef715668496692c54135f93f9433}] 2023-02-12 14:54:34,656 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0596ef715668496692c54135f93f9433 for inclusion in ac5d58614983383110f00b3bb838c006/family1 2023-02-12 14:54:34,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-12 14:54:34,658 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:34,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:34,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228/family1/0596ef715668496692c54135f93f9433 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__2enirrhd2hihprcalbodlt8atjpg2cnu7dtr5dpm5f4jtg0feaubjt1hadqkqrnm/family1/0596ef715668496692c54135f93f9433 2023-02-12 14:54:34,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__2enirrhd2hihprcalbodlt8atjpg2cnu7dtr5dpm5f4jtg0feaubjt1hadqkqrnm/family1/0596ef715668496692c54135f93f9433 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_ 2023-02-12 14:54:34,662 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__2enirrhd2hihprcalbodlt8atjpg2cnu7dtr5dpm5f4jtg0feaubjt1hadqkqrnm/family1/0596ef715668496692c54135f93f9433 into ac5d58614983383110f00b3bb838c006/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_ - updating store file list. 2023-02-12 14:54:34,666 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_ into ac5d58614983383110f00b3bb838c006/family1 2023-02-12 14:54:34,666 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__2enirrhd2hihprcalbodlt8atjpg2cnu7dtr5dpm5f4jtg0feaubjt1hadqkqrnm/family1/0596ef715668496692c54135f93f9433 into ac5d58614983383110f00b3bb838c006/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_) 2023-02-12 14:54:34,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__2enirrhd2hihprcalbodlt8atjpg2cnu7dtr5dpm5f4jtg0feaubjt1hadqkqrnm/family1/0596ef715668496692c54135f93f9433 2023-02-12 14:54:34,666 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-12 14:54:34,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51214 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-12 14:54:34,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): ac5d58614983383110f00b3bb838c006/family1 is initiating minor compaction 2023-02-12 14:54:34,669 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7a547c7e; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:34,669 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of ac5d58614983383110f00b3bb838c006/family1 in testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:54:34,669 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/24878ff5f31e4f1ba833c05f6c3e9b79, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/605b3df86de14db4b855b7e71bd4f1f7, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/9fe9f6a64a864d87a1b1ad750b21e6a0, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/25ecd6f3887544e4b66122c7a29aa23a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/ab6d13d6b57a452bbdc4a3645ac17c04, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/57090046f105441ebdd37b22a7a992a1, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/b2219a5be7674b9a802a575cc70f33a8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/60cd5061548948f1999a2940be5bcd9e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/553a6ebcd73b4167b0733d055ee2c60b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp, totalSize=50.0 K 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 24878ff5f31e4f1ba833c05f6c3e9b79, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 605b3df86de14db4b855b7e71bd4f1f7, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 9fe9f6a64a864d87a1b1ad750b21e6a0, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 25ecd6f3887544e4b66122c7a29aa23a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting ab6d13d6b57a452bbdc4a3645ac17c04, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 57090046f105441ebdd37b22a7a992a1, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting b2219a5be7674b9a802a575cc70f33a8, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-12 14:54:34,669 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 60cd5061548948f1999a2940be5bcd9e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-12 14:54:34,670 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 553a6ebcd73b4167b0733d055ee2c60b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-12 14:54:34,670 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-12 14:54:34,671 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:34,672 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:34,673 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284803a4b6bfc34c668875a5c6c48a16cb to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512284803a4b6bfc34c668875a5c6c48a16cb 2023-02-12 14:54:34,675 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512316f3f7c4255d8457bb6ae7325fed3f539 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512316f3f7c4255d8457bb6ae7325fed3f539 2023-02-12 14:54:34,676 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c0ab2fe18ff84cfe9fdd26f5c7798abf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20160101c0ab2fe18ff84cfe9fdd26f5c7798abf 2023-02-12 14:54:34,677 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601039f789372314643ffba6867d2420c23cf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201601039f789372314643ffba6867d2420c23cf 2023-02-12 14:54:34,677 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 4 expired mob files are deleted 2023-02-12 14:54:34,677 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,678 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:34,705 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): ac5d58614983383110f00b3bb838c006#family1#compaction#49 average throughput is 1.36 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:35,135 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057c25c6b1b7b540d190b51418bd0c95f2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512057c25c6b1b7b540d190b51418bd0c95f2 2023-02-12 14:54:35,135 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:35,159 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/062df4aa4c9f4e8fb0ac0edda3abda7f first=Optional[r00] last=Optional[r2] 2023-02-12 14:54:35,159 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/062df4aa4c9f4e8fb0ac0edda3abda7f}] 2023-02-12 14:54:35,161 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/062df4aa4c9f4e8fb0ac0edda3abda7f for inclusion in ac5d58614983383110f00b3bb838c006/family1 2023-02-12 14:54:35,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-12 14:54:35,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:35,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:35,165 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130/family1/062df4aa4c9f4e8fb0ac0edda3abda7f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vmscmdbth69mvnsd0snffr4m4evucf5jiqim6gouurmu4eujvi2upbnd45m73jvj/family1/062df4aa4c9f4e8fb0ac0edda3abda7f 2023-02-12 14:54:35,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vmscmdbth69mvnsd0snffr4m4evucf5jiqim6gouurmu4eujvi2upbnd45m73jvj/family1/062df4aa4c9f4e8fb0ac0edda3abda7f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/bd82b19f50da489496f2cb8113814568_SeqId_46_ 2023-02-12 14:54:35,168 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vmscmdbth69mvnsd0snffr4m4evucf5jiqim6gouurmu4eujvi2upbnd45m73jvj/family1/062df4aa4c9f4e8fb0ac0edda3abda7f into ac5d58614983383110f00b3bb838c006/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/bd82b19f50da489496f2cb8113814568_SeqId_46_ - updating store file list. 2023-02-12 14:54:35,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/bd82b19f50da489496f2cb8113814568_SeqId_46_ into ac5d58614983383110f00b3bb838c006/family1 2023-02-12 14:54:35,172 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vmscmdbth69mvnsd0snffr4m4evucf5jiqim6gouurmu4eujvi2upbnd45m73jvj/family1/062df4aa4c9f4e8fb0ac0edda3abda7f into ac5d58614983383110f00b3bb838c006/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/bd82b19f50da489496f2cb8113814568_SeqId_46_) 2023-02-12 14:54:35,172 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyPolicy__vmscmdbth69mvnsd0snffr4m4evucf5jiqim6gouurmu4eujvi2upbnd45m73jvj/family1/062df4aa4c9f4e8fb0ac0edda3abda7f 2023-02-12 14:54:35,175 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:35,175 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:35,177 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b549fad94a524b87b8d4907bfe190182 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e20151130b549fad94a524b87b8d4907bfe190182 2023-02-12 14:54:35,179 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120170fc5f0b05814968ba499b8a3874c472 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e2015120170fc5f0b05814968ba499b8a3874c472 2023-02-12 14:54:35,180 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512057b5c7476b61845d4a82c4bd6312c250d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/72ba7990731d4bd518c891adecd81f43/family1/d41d8cd98f00b204e9800998ecf8427e201512057b5c7476b61845d4a82c4bd6312c250d 2023-02-12 14:54:35,180 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyPolicy 3 expired mob files are deleted 2023-02-12 14:54:35,180 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyPolicy/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-12 14:54:35,180 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyPolicy' and column='family1' 2023-02-12 14:54:35,180 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyPolicy' column='family1' 2023-02-12 14:54:35,181 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyPolicy', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:35,181 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=38, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:54:35,183 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyPolicy.mobLock, type=EXCLUSIVE in 1.0140 sec 2023-02-12 14:54:35,282 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyPolicy Thread=257 (was 255) - Thread LEAK? -, OpenFileDescriptor=631 (was 606) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=245 (was 254), ProcessCount=168 (was 168), AvailableMemoryMB=3940 (was 4023) 2023-02-12 14:54:35,295 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=257, OpenFileDescriptor=631, MaxFileDescriptor=60000, SystemLoadAverage=245, ProcessCount=168, AvailableMemoryMB=3939 2023-02-12 14:54:35,297 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-12 14:54:35,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:35,299 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:54:35,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly" procId is: 39 2023-02-12 14:54:35,300 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:54:35,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-12 14:54:35,301 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:35,302 DEBUG [HFileArchiver-9] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 empty. 2023-02-12 14:54:35,302 DEBUG [HFileArchiver-9] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:35,302 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly regions 2023-02-12 14:54:35,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-12 14:54:35,521 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/.tmp/family1/ad5e48cd6e2a4b4b8b3fdd1c7c16a394 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/ad5e48cd6e2a4b4b8b3fdd1c7c16a394 2023-02-12 14:54:35,526 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in ac5d58614983383110f00b3bb838c006/family1 of ac5d58614983383110f00b3bb838c006 into ad5e48cd6e2a4b4b8b3fdd1c7c16a394(size=6.6 K), total size for store is 337.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:54:35,526 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for ac5d58614983383110f00b3bb838c006: 2023-02-12 14:54:35,526 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006., storeName=family1, fileCount=10, fileSize=50.0 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.5 K), priority=2, time=1676213674669; duration=0sec 2023-02-12 14:54:35,526 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:35,559 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/24878ff5f31e4f1ba833c05f6c3e9b79, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/605b3df86de14db4b855b7e71bd4f1f7, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/9fe9f6a64a864d87a1b1ad750b21e6a0, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/25ecd6f3887544e4b66122c7a29aa23a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/ab6d13d6b57a452bbdc4a3645ac17c04, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/57090046f105441ebdd37b22a7a992a1, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/b2219a5be7674b9a802a575cc70f33a8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/60cd5061548948f1999a2940be5bcd9e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/553a6ebcd73b4167b0733d055ee2c60b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_] to archive 2023-02-12 14:54:35,559 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:35,561 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/24878ff5f31e4f1ba833c05f6c3e9b79 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/24878ff5f31e4f1ba833c05f6c3e9b79 2023-02-12 14:54:35,562 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/605b3df86de14db4b855b7e71bd4f1f7 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/605b3df86de14db4b855b7e71bd4f1f7 2023-02-12 14:54:35,563 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/9fe9f6a64a864d87a1b1ad750b21e6a0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/9fe9f6a64a864d87a1b1ad750b21e6a0 2023-02-12 14:54:35,564 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/25ecd6f3887544e4b66122c7a29aa23a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/25ecd6f3887544e4b66122c7a29aa23a 2023-02-12 14:54:35,565 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/ab6d13d6b57a452bbdc4a3645ac17c04 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/ab6d13d6b57a452bbdc4a3645ac17c04 2023-02-12 14:54:35,566 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/57090046f105441ebdd37b22a7a992a1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/57090046f105441ebdd37b22a7a992a1 2023-02-12 14:54:35,567 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/b2219a5be7674b9a802a575cc70f33a8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/b2219a5be7674b9a802a575cc70f33a8 2023-02-12 14:54:35,568 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/60cd5061548948f1999a2940be5bcd9e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/60cd5061548948f1999a2940be5bcd9e 2023-02-12 14:54:35,569 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/553a6ebcd73b4167b0733d055ee2c60b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/553a6ebcd73b4167b0733d055ee2c60b 2023-02-12 14:54:35,570 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/family1/90d23e7f40b54e5dbebcb1cce6ce7622_SeqId_44_ 2023-02-12 14:54:35,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-12 14:54:35,712 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-12 14:54:35,713 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(7671): creating {ENCODED => aac84b1ca5b7fcfec0ef433f2e9b7077, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:54:35,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-12 14:54:36,119 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:36,119 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1603): Closing aac84b1ca5b7fcfec0ef433f2e9b7077, disabling compactions & flushes 2023-02-12 14:54:36,119 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:36,119 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:36,119 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. after waiting 0 ms 2023-02-12 14:54:36,119 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:36,119 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:36,119 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly-pool-0] regionserver.HRegion(1557): Region close journal for aac84b1ca5b7fcfec0ef433f2e9b7077: Waiting for close lock at 1676213676119Disabling compacts and flushes for region at 1676213676119Disabling writes for close at 1676213676119Writing region close event to WAL at 1676213676119Closed at 1676213676119 2023-02-12 14:54:36,122 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:54:36,123 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213676123"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213676123"}]},"ts":"1676213676123"} 2023-02-12 14:54:36,124 INFO [PEWorker-4] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:54:36,125 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:54:36,125 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213676125"}]},"ts":"1676213676125"} 2023-02-12 14:54:36,126 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLING in hbase:meta 2023-02-12 14:54:36,129 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, ASSIGN}] 2023-02-12 14:54:36,130 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, ASSIGN 2023-02-12 14:54:36,131 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=40, ppid=39, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:54:36,282 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:36,282 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213676282"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213676282"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213676282"}]},"ts":"1676213676282"} 2023-02-12 14:54:36,284 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=41, ppid=40, state=RUNNABLE; OpenRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:36,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-12 14:54:36,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:36,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aac84b1ca5b7fcfec0ef433f2e9b7077, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:54:36,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:36,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:36,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:36,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:36,440 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:36,442 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:36,442 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:36,442 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aac84b1ca5b7fcfec0ef433f2e9b7077 columnFamilyName family1 2023-02-12 14:54:36,443 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(310): Store=aac84b1ca5b7fcfec0ef433f2e9b7077/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:36,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:36,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:36,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:36,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:54:36,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aac84b1ca5b7fcfec0ef433f2e9b7077; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64470791, jitterRate=-0.03931035101413727}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:54:36,449 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aac84b1ca5b7fcfec0ef433f2e9b7077: Running coprocessor pre-open hook at 1676213676439Writing region info on filesystem at 1676213676439Initializing all the Stores at 1676213676440 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676213676440Cleaning up temporary data from old regions at 1676213676445 (+5 ms)Cleaning up detritus from prior splits at 1676213676445Running coprocessor post-open hooks at 1676213676449 (+4 ms)Region opened successfully at 1676213676449 2023-02-12 14:54:36,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., pid=41, masterSystemTime=1676213676436 2023-02-12 14:54:36,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:36,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:36,451 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=40 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:36,451 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213676451"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213676451"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213676451"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213676451"}]},"ts":"1676213676451"} 2023-02-12 14:54:36,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=41, resume processing ppid=40 2023-02-12 14:54:36,454 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, ppid=40, state=SUCCESS; OpenRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001 in 169 msec 2023-02-12 14:54:36,455 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=40, resume processing ppid=39 2023-02-12 14:54:36,456 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=40, ppid=39, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, ASSIGN in 325 msec 2023-02-12 14:54:36,456 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:54:36,456 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213676456"}]},"ts":"1676213676456"} 2023-02-12 14:54:36,457 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, state=ENABLED in hbase:meta 2023-02-12 14:54:36,459 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=39, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:54:36,461 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=39, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.1630 sec 2023-02-12 14:54:37,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=39 2023-02-12 14:54:37,405 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, procId: 39 completed 2023-02-12 14:54:37,425 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:37,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,427 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:37,427 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:37,427 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:37,427 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:37,428 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,428 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,430 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:37,430 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:37,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:37,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:37,430 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:37,430 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:37,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:37,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,431 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,431 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:37,431 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:37,431 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:37,431 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:37,432 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:37,432 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:37,432 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:37,432 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:37,432 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-12 14:54:37,849 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d6438067307a4c4ebc7d814c338878ee to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130d6438067307a4c4ebc7d814c338878ee 2023-02-12 14:54:38,126 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:38,169 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 14:54:38,252 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/a8623c5a1f854aeeb99b8f91b8a036c6 2023-02-12 14:54:38,258 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/a8623c5a1f854aeeb99b8f91b8a036c6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a8623c5a1f854aeeb99b8f91b8a036c6 2023-02-12 14:54:38,263 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a8623c5a1f854aeeb99b8f91b8a036c6, entries=1000, sequenceid=5, filesize=191.4 K 2023-02-12 14:54:38,263 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 831ms, sequenceid=5, compaction requested=false 2023-02-12 14:54:38,264 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:38,264 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:38,264 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:38,266 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,266 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:38,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:38,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:38,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:38,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:38,268 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4eff929a[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:38,268 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:38,268 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,269 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,269 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,269 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:38,269 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:38,269 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:38,269 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,269 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:38,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:38,271 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:38,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:38,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:38,273 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:38,273 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:38,273 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:38,273 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:38,273 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,273 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:38,275 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,275 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:38,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,275 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:38,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:38,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:38,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,277 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:38,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,279 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:38,279 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:38,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:38,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:38,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:38,279 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:38,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:38,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:38,380 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:38,380 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:38,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:38,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:38,386 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,386 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:38,386 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:38,386 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:38,386 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:38,386 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,386 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,387 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:38,387 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:38,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:38,388 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:38,388 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:38,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:38,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:38,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:38,389 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:38,389 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:38,389 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:38,389 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:38,389 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:38,389 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:38,389 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:38,389 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-12 14:54:38,802 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128902810d65da04a68b14b4612ea5f54e1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128902810d65da04a68b14b4612ea5f54e1 2023-02-12 14:54:39,208 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/c222d6b4f3ca4440b1adc8b3d1c7e516 2023-02-12 14:54:39,213 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/c222d6b4f3ca4440b1adc8b3d1c7e516 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/c222d6b4f3ca4440b1adc8b3d1c7e516 2023-02-12 14:54:39,217 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/c222d6b4f3ca4440b1adc8b3d1c7e516, entries=1, sequenceid=9, filesize=5.0 K 2023-02-12 14:54:39,218 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 829ms, sequenceid=9, compaction requested=false 2023-02-12 14:54:39,218 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:39,218 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:39,218 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:39,218 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:39,218 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,219 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:39,219 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:39,221 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:39,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,221 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:39,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:39,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:39,223 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@378048b9[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:39,223 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:39,223 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,224 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:39,224 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:39,224 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:39,224 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,224 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:39,225 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:39,225 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:39,225 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,225 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:39,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:39,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:39,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:39,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:39,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:39,228 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:39,228 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:39,228 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:39,228 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:39,228 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,228 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:39,229 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,229 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:39,229 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,229 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:39,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:39,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:39,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:39,230 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:39,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:39,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,231 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:39,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:39,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:39,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:39,237 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-12 14:54:39,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:39,237 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:39,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:39,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,237 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:39,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:39,238 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:39,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:39,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,238 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:39,338 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:39,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:39,343 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:39,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:39,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,344 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:39,344 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:39,344 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:39,344 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:39,345 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,345 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,346 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:39,346 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:39,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:39,346 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:39,346 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:39,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,347 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,347 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:39,347 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:39,347 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:39,347 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:39,347 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:39,347 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:39,348 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:39,348 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:39,348 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:39,348 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:39,760 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151201632c75f409d049ccb5a9adc9924ceb9a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201632c75f409d049ccb5a9adc9924ceb9a 2023-02-12 14:54:40,166 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/b838e6036f194ea0af127b6af79e99cf 2023-02-12 14:54:40,171 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/b838e6036f194ea0af127b6af79e99cf as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/b838e6036f194ea0af127b6af79e99cf 2023-02-12 14:54:40,175 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/b838e6036f194ea0af127b6af79e99cf, entries=1, sequenceid=13, filesize=5.0 K 2023-02-12 14:54:40,176 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 828ms, sequenceid=13, compaction requested=true 2023-02-12 14:54:40,176 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:40,176 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:40,176 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:40,176 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:40,176 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,176 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:40,176 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:40,177 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,178 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:40,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:40,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:40,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2aac26d1[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:40,179 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:40,179 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,180 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,180 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:40,180 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:40,181 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:40,181 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,181 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:40,183 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:40,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:40,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:40,183 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:40,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:40,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:40,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:40,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:40,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:40,186 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:40,186 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:40,186 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:40,186 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,186 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:40,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:40,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:40,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:40,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,189 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:40,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:40,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:40,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:40,191 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:40,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:40,191 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:40,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:40,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:40,292 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:40,292 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:40,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:40,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:40,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,298 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:40,298 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:40,298 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:40,298 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:40,298 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,299 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,300 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:40,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:40,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:40,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:40,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:40,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,300 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:40,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,301 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,301 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:40,301 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:40,301 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:40,301 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:40,301 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:40,301 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:40,301 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:40,301 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:40,301 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:40,301 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:40,715 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512057a332fbe9c4842548823993656031111 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512057a332fbe9c4842548823993656031111 2023-02-12 14:54:41,121 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/0c0419823b724bf4a162df70b83b9604 2023-02-12 14:54:41,125 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/0c0419823b724bf4a162df70b83b9604 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/0c0419823b724bf4a162df70b83b9604 2023-02-12 14:54:41,129 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/0c0419823b724bf4a162df70b83b9604, entries=1, sequenceid=17, filesize=5.0 K 2023-02-12 14:54:41,130 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 829ms, sequenceid=17, compaction requested=true 2023-02-12 14:54:41,130 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:41,130 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:41,130 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:41,134 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,134 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:41,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:41,134 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,134 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:41,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:41,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:41,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:41,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:41,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15c6c47e[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:41,135 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:41,135 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,136 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,136 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:41,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:41,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:41,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:41,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:41,137 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,137 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:41,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:41,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:41,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:41,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:41,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:41,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:41,140 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:41,140 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:41,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:41,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:41,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,140 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:41,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,141 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:41,141 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:41,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:41,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:41,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:41,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:41,145 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:41,146 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:41,145 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:41,145 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:41,146 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,146 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,146 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,146 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:41,146 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:41,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:41,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:41,246 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:41,247 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:41,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:41,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:41,253 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,253 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:41,253 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:41,253 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:41,253 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:41,253 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,253 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,255 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:41,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:41,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:41,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:41,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,255 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,256 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:41,256 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:41,256 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:41,275 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:41,275 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:41,275 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:41,275 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:54:41,275 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3447f765 to 127.0.0.1:61565 2023-02-12 14:54:41,275 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:54:41,276 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:54:41,276 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2e5e3de1 to 127.0.0.1:61565 2023-02-12 14:54:41,276 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:54:41,276 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:54:41,276 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x7b34e372 to 127.0.0.1:61565 2023-02-12 14:54:41,276 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:54:41,693 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151228a0604c3c0836421597b3443ea99f01ec to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228a0604c3c0836421597b3443ea99f01ec 2023-02-12 14:54:41,975 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:54:42,098 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/96d0277ee46948a888790e79ff1cca2a 2023-02-12 14:54:42,103 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/96d0277ee46948a888790e79ff1cca2a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/96d0277ee46948a888790e79ff1cca2a 2023-02-12 14:54:42,108 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/96d0277ee46948a888790e79ff1cca2a, entries=1, sequenceid=21, filesize=5.0 K 2023-02-12 14:54:42,109 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 834ms, sequenceid=21, compaction requested=true 2023-02-12 14:54:42,109 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:42,109 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:42,109 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:42,109 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:42,109 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,109 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:42,109 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:42,114 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,114 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:42,114 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,114 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:42,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:42,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:42,116 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@271fff1e[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:42,116 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:42,116 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,117 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,117 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,117 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:42,117 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:42,117 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:42,117 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,117 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:42,118 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,118 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:42,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:42,118 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:42,118 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:42,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:42,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:42,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:42,120 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:42,120 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:42,121 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:42,121 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:42,121 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,121 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:42,123 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,123 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,123 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,123 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:42,123 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,124 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:42,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:42,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,125 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:42,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,127 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:42,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly because node does not exist (not an error) 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:42,127 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:42,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,127 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,128 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:42,128 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:42,128 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:42,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:42,228 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:42,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:42,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:42,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:42,234 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,234 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:42,234 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:42,234 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:42,234 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:42,234 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,234 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,235 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,235 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:42,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:42,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:42,236 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:42,236 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:42,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:42,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:42,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:42,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:42,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:42,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:42,237 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:42,649 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231f94399eb805440e38c47a5e5acab0e3c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231f94399eb805440e38c47a5e5acab0e3c 2023-02-12 14:54:43,055 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/51ee273389cd4d949199c345821c0333 2023-02-12 14:54:43,059 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/51ee273389cd4d949199c345821c0333 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/51ee273389cd4d949199c345821c0333 2023-02-12 14:54:43,063 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/51ee273389cd4d949199c345821c0333, entries=1, sequenceid=25, filesize=5.0 K 2023-02-12 14:54:43,064 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 827ms, sequenceid=25, compaction requested=true 2023-02-12 14:54:43,064 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:43,064 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:43,064 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:43,064 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:43,064 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,064 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:43,064 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:43,065 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,065 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:43,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:43,066 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,066 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:43,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:43,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:43,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:43,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:43,067 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@126fbec4[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:43,067 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:43,067 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,069 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,069 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:43,069 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,069 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:43,069 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:43,069 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:43,070 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:43,070 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,070 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:43,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:43,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:43,070 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:43,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:43,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:43,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:43,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:43,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:43,072 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:43,072 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:43,072 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:43,072 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,072 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:43,073 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,073 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:43,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,074 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:43,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:43,074 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:43,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,075 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:43,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:43,077 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:43,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:43,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:43,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:43,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:43,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:43,077 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:43,178 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:43,178 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:43,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:43,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:43,184 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,184 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:43,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:43,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:43,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:43,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,191 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:43,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:43,191 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:43,191 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:43,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:43,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,192 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,192 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:43,192 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:43,192 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:43,192 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:43,192 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:43,192 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:43,192 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:43,192 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:43,192 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:43,193 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:43,604 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601015c4a1348895543bd96f20a4480188bbd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601015c4a1348895543bd96f20a4480188bbd 2023-02-12 14:54:44,009 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/f90895883a6346c6ab9e234d7bbfa5c4 2023-02-12 14:54:44,014 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/f90895883a6346c6ab9e234d7bbfa5c4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/f90895883a6346c6ab9e234d7bbfa5c4 2023-02-12 14:54:44,018 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/f90895883a6346c6ab9e234d7bbfa5c4, entries=1, sequenceid=29, filesize=5.0 K 2023-02-12 14:54:44,018 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 825ms, sequenceid=29, compaction requested=true 2023-02-12 14:54:44,018 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:44,018 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:44,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:44,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:44,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:44,019 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:44,020 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,020 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:44,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:44,020 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,020 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:44,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:44,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:44,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@59be98f6[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:44,022 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:44,022 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,023 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,023 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,023 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:44,023 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:44,023 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:44,023 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,023 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:44,024 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,024 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:44,024 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:44,024 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:44,026 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:44,026 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:44,026 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:44,026 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,026 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:44,030 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,030 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,030 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:44,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:44,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:44,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:44,031 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,031 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:44,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:44,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:44,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,032 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:44,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:44,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:44,034 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:44,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:44,034 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:44,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:44,034 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,134 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:44,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:44,139 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:44,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:44,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,140 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:44,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:44,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:44,140 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:44,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,142 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:44,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:44,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:44,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:44,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,143 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,143 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:44,143 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:44,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:44,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:44,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:44,143 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:44,144 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:44,144 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:44,144 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:44,555 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601037c5feca0a6e4477f8af2e4c7e0ba4b07 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601037c5feca0a6e4477f8af2e4c7e0ba4b07 2023-02-12 14:54:44,960 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/32e32a14fa824a66b80bd9e4f7ecb4ec 2023-02-12 14:54:44,965 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/32e32a14fa824a66b80bd9e4f7ecb4ec as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/32e32a14fa824a66b80bd9e4f7ecb4ec 2023-02-12 14:54:44,969 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/32e32a14fa824a66b80bd9e4f7ecb4ec, entries=1, sequenceid=33, filesize=5.0 K 2023-02-12 14:54:44,970 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 826ms, sequenceid=33, compaction requested=true 2023-02-12 14:54:44,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:44,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:44,970 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:44,970 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:44,970 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,970 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:44,970 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:44,972 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,972 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:44,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:44,973 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,973 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:44,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:44,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:44,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:44,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:44,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7af8528b[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:44,974 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:44,974 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,977 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,977 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:44,977 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:44,977 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:44,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,977 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:44,978 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,978 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:44,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:44,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:44,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:44,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:44,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:44,980 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:44,980 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:44,981 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:44,981 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:44,981 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,981 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:44,982 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,982 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:44,982 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:44,982 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:44,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,983 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:44,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,985 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,985 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:44,985 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:44,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:44,985 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:44,985 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,985 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:44,985 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:44,985 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:44,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:44,985 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,986 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:44,986 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:44,986 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,986 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:44,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:44,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:45,086 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:45,087 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:45,090 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:45,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:45,092 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,092 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:45,092 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:45,092 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:45,092 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:45,093 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,093 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,094 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:45,094 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:45,094 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,094 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:45,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,094 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:45,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,095 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,095 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:45,095 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,095 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:45,095 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:45,095 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:45,095 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:45,095 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:45,095 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:45,095 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:45,095 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:45,508 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015090795f3e1f48d4d41259e48dff2fd64a315 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e2015090795f3e1f48d4d41259e48dff2fd64a315 2023-02-12 14:54:45,913 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/7efd3d1ea9a14512868d7cd67b03b16a 2023-02-12 14:54:45,918 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/7efd3d1ea9a14512868d7cd67b03b16a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/7efd3d1ea9a14512868d7cd67b03b16a 2023-02-12 14:54:45,922 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/7efd3d1ea9a14512868d7cd67b03b16a, entries=1, sequenceid=37, filesize=5.0 K 2023-02-12 14:54:45,923 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 828ms, sequenceid=37, compaction requested=true 2023-02-12 14:54:45,923 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:45,923 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:45,923 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:45,926 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,926 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:45,926 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,926 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:45,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:45,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:45,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:45,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@22c7b5c5[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:45,927 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:45,927 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,928 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,928 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:45,928 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:45,928 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:45,929 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,929 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:45,930 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,930 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:45,930 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:45,930 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:45,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:45,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:45,932 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:45,932 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:45,932 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:45,932 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:45,932 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,932 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:45,934 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,934 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,934 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:45,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:45,934 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,934 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:45,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,935 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:45,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,942 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:45,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:45,942 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:45,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:45,942 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:45,943 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:45,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:45,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:46,043 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:46,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:46,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:46,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:46,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,049 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:46,049 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:46,049 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' 2023-02-12 14:54:46,049 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', kicking off acquire phase on members. 2023-02-12 14:54:46,050 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,050 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,051 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,051 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:46,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:46,051 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,051 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:46,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly is still present, removing 2023-02-12 14:54:46,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' with timeout 60000ms 2023-02-12 14:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'acquire' stage 2023-02-12 14:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:46,052 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:46,053 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:46,053 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. started... 2023-02-12 14:54:46,053 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing aac84b1ca5b7fcfec0ef433f2e9b7077 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:46,464 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120a7f0c01b3ed04d1ea419403651defa33 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120a7f0c01b3ed04d1ea419403651defa33 2023-02-12 14:54:46,869 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/8b3e7b7b321b47b3ae27366b8da98cf0 2023-02-12 14:54:46,874 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/8b3e7b7b321b47b3ae27366b8da98cf0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/8b3e7b7b321b47b3ae27366b8da98cf0 2023-02-12 14:54:46,877 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/8b3e7b7b321b47b3ae27366b8da98cf0, entries=1, sequenceid=41, filesize=5.0 K 2023-02-12 14:54:46,878 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for aac84b1ca5b7fcfec0ef433f2e9b7077 in 825ms, sequenceid=41, compaction requested=true 2023-02-12 14:54:46,878 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:46,878 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:46,878 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:46,878 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:46,878 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,878 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally acquired 2023-02-12 14:54:46,878 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly) in zk 2023-02-12 14:54:46,880 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,880 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:46,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:46,880 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,880 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:46,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:46,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' on coordinator 2023-02-12 14:54:46,881 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4db1a6c7[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:46,881 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' starting 'in-barrier' execution. 2023-02-12 14:54:46,882 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,882 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' received 'reached' from coordinator. 2023-02-12 14:54:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' locally completed 2023-02-12 14:54:46,883 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:46,883 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,883 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:46,885 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' has notified controller of completion 2023-02-12 14:54:46,885 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:46,885 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,885 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' completed. 2023-02-12 14:54:46,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:46,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:46,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:46,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:46,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:46,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:46,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', counting down latch. Waiting for 0 more 2023-02-12 14:54:46,887 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' execution completed 2023-02-12 14:54:46,887 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:46,887 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:46,887 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,887 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeeklyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:46,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:46,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:46,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:46,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:46,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:46,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:46,889 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:46,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,890 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:46,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:46,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:46,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:46,892 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:46,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:46,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:46,892 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:46,892 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,993 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:46,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:46,995 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:46,996 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:46,997 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=42, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:46,998 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:46,998 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=42, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214286998 2023-02-12 14:54:47,000 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:54:47,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-12 14:54:47,001 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:47,001 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x4ba3f36e to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:54:47,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2ee41e94, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:54:47,005 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,847 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103df4f675e669a491b991ac325a2884157 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20160103df4f675e669a491b991ac325a2884157 2023-02-12 14:54:47,847 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:47,860 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:54:47,861 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56966, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:54:47,867 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:54:47,868 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35534, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:54:47,877 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc49a36734bb40f0a699fb001ab20c3f first=Optional[r3] last=Optional[r6] 2023-02-12 14:54:47,877 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc49a36734bb40f0a699fb001ab20c3f}] 2023-02-12 14:54:47,882 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc49a36734bb40f0a699fb001ab20c3f for inclusion in aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:47,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-12 14:54:47,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:47,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:47,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228/family1/bc49a36734bb40f0a699fb001ab20c3f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__k88jktk4lnpnuddlh75isa9lf704bsi9gob2pv5hdr6u64v6cdk5rkvkq3bua6n8/family1/bc49a36734bb40f0a699fb001ab20c3f 2023-02-12 14:54:47,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__k88jktk4lnpnuddlh75isa9lf704bsi9gob2pv5hdr6u64v6cdk5rkvkq3bua6n8/family1/bc49a36734bb40f0a699fb001ab20c3f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_ 2023-02-12 14:54:47,887 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__k88jktk4lnpnuddlh75isa9lf704bsi9gob2pv5hdr6u64v6cdk5rkvkq3bua6n8/family1/bc49a36734bb40f0a699fb001ab20c3f into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_ - updating store file list. 2023-02-12 14:54:47,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_ into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:47,890 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__k88jktk4lnpnuddlh75isa9lf704bsi9gob2pv5hdr6u64v6cdk5rkvkq3bua6n8/family1/bc49a36734bb40f0a699fb001ab20c3f into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_) 2023-02-12 14:54:47,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__k88jktk4lnpnuddlh75isa9lf704bsi9gob2pv5hdr6u64v6cdk5rkvkq3bua6n8/family1/bc49a36734bb40f0a699fb001ab20c3f 2023-02-12 14:54:47,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-12 14:54:47,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51565 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-12 14:54:47,893 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): aac84b1ca5b7fcfec0ef433f2e9b7077/family1 is initiating minor compaction 2023-02-12 14:54:47,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@711e3d45; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:47,894 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of aac84b1ca5b7fcfec0ef433f2e9b7077/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:47,894 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/c222d6b4f3ca4440b1adc8b3d1c7e516, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/b838e6036f194ea0af127b6af79e99cf, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/0c0419823b724bf4a162df70b83b9604, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/96d0277ee46948a888790e79ff1cca2a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/51ee273389cd4d949199c345821c0333, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/f90895883a6346c6ab9e234d7bbfa5c4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/32e32a14fa824a66b80bd9e4f7ecb4ec, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/7efd3d1ea9a14512868d7cd67b03b16a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/8b3e7b7b321b47b3ae27366b8da98cf0, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp, totalSize=50.4 K 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting c222d6b4f3ca4440b1adc8b3d1c7e516, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting b838e6036f194ea0af127b6af79e99cf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 0c0419823b724bf4a162df70b83b9604, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 96d0277ee46948a888790e79ff1cca2a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 51ee273389cd4d949199c345821c0333, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting f90895883a6346c6ab9e234d7bbfa5c4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 32e32a14fa824a66b80bd9e4f7ecb4ec, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 7efd3d1ea9a14512868d7cd67b03b16a, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 8b3e7b7b321b47b3ae27366b8da98cf0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-12 14:54:47,894 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting a233c6a54dc342c7ba3fbec128f446de_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-12 14:54:47,895 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:47,896 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:47,898 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228a0604c3c0836421597b3443ea99f01ec to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151228a0604c3c0836421597b3443ea99f01ec 2023-02-12 14:54:47,899 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231f94399eb805440e38c47a5e5acab0e3c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151231f94399eb805440e38c47a5e5acab0e3c 2023-02-12 14:54:47,900 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601015c4a1348895543bd96f20a4480188bbd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601015c4a1348895543bd96f20a4480188bbd 2023-02-12 14:54:47,901 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601037c5feca0a6e4477f8af2e4c7e0ba4b07 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201601037c5feca0a6e4477f8af2e4c7e0ba4b07 2023-02-12 14:54:47,901 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 4 expired mob files are deleted 2023-02-12 14:54:47,901 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,902 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:47,923 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): aac84b1ca5b7fcfec0ef433f2e9b7077#family1#compaction#60 average throughput is 1.59 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:48,747 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/3609931dc7754118bc9df4162d9e4386 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/3609931dc7754118bc9df4162d9e4386 2023-02-12 14:54:48,753 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in aac84b1ca5b7fcfec0ef433f2e9b7077/family1 of aac84b1ca5b7fcfec0ef433f2e9b7077 into 3609931dc7754118bc9df4162d9e4386(size=6.8 K), total size for store is 198.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:54:48,753 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:48,753 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., storeName=family1, fileCount=10, fileSize=50.4 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676213687893; duration=0sec 2023-02-12 14:54:48,753 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:48,768 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/c222d6b4f3ca4440b1adc8b3d1c7e516, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/b838e6036f194ea0af127b6af79e99cf, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/0c0419823b724bf4a162df70b83b9604, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/96d0277ee46948a888790e79ff1cca2a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/51ee273389cd4d949199c345821c0333, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/f90895883a6346c6ab9e234d7bbfa5c4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/32e32a14fa824a66b80bd9e4f7ecb4ec, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/7efd3d1ea9a14512868d7cd67b03b16a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/8b3e7b7b321b47b3ae27366b8da98cf0, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_] to archive 2023-02-12 14:54:48,769 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:48,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/c222d6b4f3ca4440b1adc8b3d1c7e516 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/c222d6b4f3ca4440b1adc8b3d1c7e516 2023-02-12 14:54:48,771 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205818bbb570a7549cd97bed17529e18e56 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151205818bbb570a7549cd97bed17529e18e56 2023-02-12 14:54:48,772 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/b838e6036f194ea0af127b6af79e99cf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/b838e6036f194ea0af127b6af79e99cf 2023-02-12 14:54:48,772 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:48,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/0c0419823b724bf4a162df70b83b9604 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/0c0419823b724bf4a162df70b83b9604 2023-02-12 14:54:48,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/96d0277ee46948a888790e79ff1cca2a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/96d0277ee46948a888790e79ff1cca2a 2023-02-12 14:54:48,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/51ee273389cd4d949199c345821c0333 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/51ee273389cd4d949199c345821c0333 2023-02-12 14:54:48,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/f90895883a6346c6ab9e234d7bbfa5c4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/f90895883a6346c6ab9e234d7bbfa5c4 2023-02-12 14:54:48,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/32e32a14fa824a66b80bd9e4f7ecb4ec to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/32e32a14fa824a66b80bd9e4f7ecb4ec 2023-02-12 14:54:48,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/7efd3d1ea9a14512868d7cd67b03b16a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/7efd3d1ea9a14512868d7cd67b03b16a 2023-02-12 14:54:48,779 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/8b3e7b7b321b47b3ae27366b8da98cf0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/8b3e7b7b321b47b3ae27366b8da98cf0 2023-02-12 14:54:48,781 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a233c6a54dc342c7ba3fbec128f446de_SeqId_44_ 2023-02-12 14:54:48,796 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/916ea6e3877b4738857cfaeb8995f217 first=Optional[r00] last=Optional[r2] 2023-02-12 14:54:48,796 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/916ea6e3877b4738857cfaeb8995f217}] 2023-02-12 14:54:48,798 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/916ea6e3877b4738857cfaeb8995f217 for inclusion in aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:48,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-12 14:54:48,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:48,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:48,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130/family1/916ea6e3877b4738857cfaeb8995f217 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6jrdjbna93aur2pmttpompepjv097ddvi5em04ja7f11asdq5ammcokodtgpvs6q/family1/916ea6e3877b4738857cfaeb8995f217 2023-02-12 14:54:48,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6jrdjbna93aur2pmttpompepjv097ddvi5em04ja7f11asdq5ammcokodtgpvs6q/family1/916ea6e3877b4738857cfaeb8995f217 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_ 2023-02-12 14:54:48,805 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6jrdjbna93aur2pmttpompepjv097ddvi5em04ja7f11asdq5ammcokodtgpvs6q/family1/916ea6e3877b4738857cfaeb8995f217 into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_ - updating store file list. 2023-02-12 14:54:48,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_ into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:48,809 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6jrdjbna93aur2pmttpompepjv097ddvi5em04ja7f11asdq5ammcokodtgpvs6q/family1/916ea6e3877b4738857cfaeb8995f217 into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_) 2023-02-12 14:54:48,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__6jrdjbna93aur2pmttpompepjv097ddvi5em04ja7f11asdq5ammcokodtgpvs6q/family1/916ea6e3877b4738857cfaeb8995f217 2023-02-12 14:54:48,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-12 14:54:48,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 399433 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-12 14:54:48,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): aac84b1ca5b7fcfec0ef433f2e9b7077/family1 is initiating minor compaction (all files) 2023-02-12 14:54:48,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@53be05bf; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:48,811 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of aac84b1ca5b7fcfec0ef433f2e9b7077/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:48,811 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a8623c5a1f854aeeb99b8f91b8a036c6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/3609931dc7754118bc9df4162d9e4386, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp, totalSize=390.1 K 2023-02-12 14:54:48,811 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting a8623c5a1f854aeeb99b8f91b8a036c6, keycount=1000, bloomtype=ROW, size=191.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840591 2023-02-12 14:54:48,812 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 3609931dc7754118bc9df4162d9e4386, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240591 2023-02-12 14:54:48,812 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 29cf8ded720142769235d99134530b6d_SeqId_47_, keycount=1002, bloomtype=ROW, size=191.8 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840591 2023-02-12 14:54:48,812 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:48,813 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:48,815 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130d6438067307a4c4ebc7d814c338878ee to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151130d6438067307a4c4ebc7d814c338878ee 2023-02-12 14:54:48,816 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201632c75f409d049ccb5a9adc9924ceb9a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151201632c75f409d049ccb5a9adc9924ceb9a 2023-02-12 14:54:48,817 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512057a332fbe9c4842548823993656031111 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201512057a332fbe9c4842548823993656031111 2023-02-12 14:54:48,817 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 3 expired mob files are deleted 2023-02-12 14:54:48,817 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-12 14:54:48,817 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:48,817 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:48,818 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:48,819 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=42, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:48,820 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=42, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 1.8240 sec 2023-02-12 14:54:48,827 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): aac84b1ca5b7fcfec0ef433f2e9b7077#family1#compaction#61 average throughput is 30.10 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:54:48,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-12 14:54:48,892 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=43, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:49,301 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-12 14:54:49,302 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000001 2023-02-12 14:54:49,302 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-12 14:54:49,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=44, ppid=43, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-12 14:54:49,307 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE}] 2023-02-12 14:54:49,308 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE 2023-02-12 14:54:49,308 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:49,308 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213689308"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213689308"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213689308"}]},"ts":"1676213689308"} 2023-02-12 14:54:49,310 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:49,461 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:49,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aac84b1ca5b7fcfec0ef433f2e9b7077, disabling compactions & flushes 2023-02-12 14:54:49,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1857): waiting for 1 compactions to complete for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:49,890 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xae3002e764b8fc20: from storage DS-7dc54834-1322-4b71-9d06-301433572c1c node DatanodeRegistration(127.0.0.1:44065, datanodeUuid=2330fbb2-a4ad-4699-a334-233f2dae637e, infoPort=44185, infoSecurePort=0, ipcPort=37945, storageInfo=lv=-57;cid=testClusterID;nsid=123215367;c=1676213612779), blocks: 57, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 1 2023-02-12 14:54:49,891 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xae3002e764b8fc20: from storage DS-b3393f4f-3a37-4e8c-9afd-5c158164d144 node DatanodeRegistration(127.0.0.1:44065, datanodeUuid=2330fbb2-a4ad-4699-a334-233f2dae637e, infoPort=44185, infoSecurePort=0, ipcPort=37945, storageInfo=lv=-57;cid=testClusterID;nsid=123215367;c=1676213612779), blocks: 53, hasStaleStorage: false, processing time: 1 msecs, invalidatedBlocks: 2 2023-02-12 14:54:50,044 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/.tmp/family1/515146fdc6304149b3feba42ae94eea0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/515146fdc6304149b3feba42ae94eea0 2023-02-12 14:54:50,048 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in aac84b1ca5b7fcfec0ef433f2e9b7077/family1 of aac84b1ca5b7fcfec0ef433f2e9b7077 into 515146fdc6304149b3feba42ae94eea0(size=193.0 K), total size for store is 193.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:54:50,048 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:50,048 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:50,048 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., storeName=family1, fileCount=3, fileSize=390.1 K (191.4 K, 6.8 K, 191.8 K), priority=2, time=1676213688811; duration=1sec 2023-02-12 14:54:50,048 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:50,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. after waiting 0 ms 2023-02-12 14:54:50,049 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:54:50,049 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:50,050 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a8623c5a1f854aeeb99b8f91b8a036c6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/3609931dc7754118bc9df4162d9e4386, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_] to archive 2023-02-12 14:54:50,051 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:50,052 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a8623c5a1f854aeeb99b8f91b8a036c6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/a8623c5a1f854aeeb99b8f91b8a036c6 2023-02-12 14:54:50,053 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/3609931dc7754118bc9df4162d9e4386 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/3609931dc7754118bc9df4162d9e4386 2023-02-12 14:54:50,054 DEBUG [StoreCloser-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/29cf8ded720142769235d99134530b6d_SeqId_47_ 2023-02-12 14:54:50,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-12 14:54:50,059 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:50,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aac84b1ca5b7fcfec0ef433f2e9b7077: Waiting for close lock at 1676213689461Running coprocessor pre-close hooks at 1676213689461Disabling compacts and flushes for region at 1676213689461Disabling writes for close at 1676213690049 (+588 ms)Writing region close event to WAL at 1676213690056 (+7 ms)Running coprocessor post-close hooks at 1676213690059 (+3 ms)Closed at 1676213690059 2023-02-12 14:54:50,059 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: aac84b1ca5b7fcfec0ef433f2e9b7077 to self. 2023-02-12 14:54:50,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,061 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=CLOSED 2023-02-12 14:54:50,061 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213690061"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213690061"}]},"ts":"1676213690061"} 2023-02-12 14:54:50,064 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-12 14:54:50,064 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001 in 753 msec 2023-02-12 14:54:50,064 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=true 2023-02-12 14:54:50,215 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:50,215 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213690214"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213690214"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213690214"}]},"ts":"1676213690214"} 2023-02-12 14:54:50,217 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=47, ppid=45, state=RUNNABLE; OpenRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:50,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:50,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aac84b1ca5b7fcfec0ef433f2e9b7077, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:54:50,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:50,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,373 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,373 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:50,374 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:50,374 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aac84b1ca5b7fcfec0ef433f2e9b7077 columnFamilyName family1 2023-02-12 14:54:50,378 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/515146fdc6304149b3feba42ae94eea0 2023-02-12 14:54:50,379 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(310): Store=aac84b1ca5b7fcfec0ef433f2e9b7077/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:50,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:50,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aac84b1ca5b7fcfec0ef433f2e9b7077; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72836608, jitterRate=0.08535003662109375}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:54:50,384 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aac84b1ca5b7fcfec0ef433f2e9b7077: Running coprocessor pre-open hook at 1676213690372Writing region info on filesystem at 1676213690372Initializing all the Stores at 1676213690372Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676213690372Cleaning up temporary data from old regions at 1676213690381 (+9 ms)Cleaning up detritus from prior splits at 1676213690381Running coprocessor post-open hooks at 1676213690384 (+3 ms)Region opened successfully at 1676213690384 2023-02-12 14:54:50,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., pid=47, masterSystemTime=1676213690368 2023-02-12 14:54:50,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:50,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:50,386 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:50,386 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213690386"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213690386"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213690386"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213690386"}]},"ts":"1676213690386"} 2023-02-12 14:54:50,389 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=47, resume processing ppid=45 2023-02-12 14:54:50,389 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=47, ppid=45, state=SUCCESS; OpenRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001 in 170 msec 2023-02-12 14:54:50,390 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-12 14:54:50,390 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE in 1.0820 sec 2023-02-12 14:54:50,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=44, resume processing ppid=43 2023-02-12 14:54:50,392 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=44, ppid=43, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.0870 sec 2023-02-12 14:54:50,393 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=43, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 1.5010 sec 2023-02-12 14:54:50,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=43 2023-02-12 14:54:50,395 INFO [Listener at localhost/37945] compactions.TestMobCompactor(278): alter status finished 2023-02-12 14:54:50,396 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:50,397 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:50,397 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=48, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:50,398 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:50,398 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=48, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214290398 2023-02-12 14:54:50,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:54:50,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-12 14:54:50,399 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:50,400 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x485ae223 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:54:50,405 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@46445502, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:54:50,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:50,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:50,405 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:50,406 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:50,406 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:51,235 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511283b3584b2b55846898c4859a6a1ea0c20 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e201511283b3584b2b55846898c4859a6a1ea0c20 2023-02-12 14:54:51,236 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:51,249 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:54:51,250 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:56982, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:54:51,256 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:54:51,258 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:35544, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:54:51,266 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f775ca90473b495b85a909c1de615bc0 first=Optional[r05] last=Optional[r8] 2023-02-12 14:54:51,267 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f775ca90473b495b85a909c1de615bc0}] 2023-02-12 14:54:51,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f775ca90473b495b85a909c1de615bc0 for inclusion in aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:51,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-12 14:54:51,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:54:51,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for aac84b1ca5b7fcfec0ef433f2e9b7077: 2023-02-12 14:54:51,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f775ca90473b495b85a909c1de615bc0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g01r82tnvqn88um75mdircnc0tph0im836i805622kelvqlos99be6u2jf2k437s/family1/f775ca90473b495b85a909c1de615bc0 2023-02-12 14:54:51,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g01r82tnvqn88um75mdircnc0tph0im836i805622kelvqlos99be6u2jf2k437s/family1/f775ca90473b495b85a909c1de615bc0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/fd8beb62343d4803babfa95006912d89_SeqId_53_ 2023-02-12 14:54:51,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g01r82tnvqn88um75mdircnc0tph0im836i805622kelvqlos99be6u2jf2k437s/family1/f775ca90473b495b85a909c1de615bc0 into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/fd8beb62343d4803babfa95006912d89_SeqId_53_ - updating store file list. 2023-02-12 14:54:51,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/fd8beb62343d4803babfa95006912d89_SeqId_53_ into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:51,281 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g01r82tnvqn88um75mdircnc0tph0im836i805622kelvqlos99be6u2jf2k437s/family1/f775ca90473b495b85a909c1de615bc0 into aac84b1ca5b7fcfec0ef433f2e9b7077/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/fd8beb62343d4803babfa95006912d89_SeqId_53_) 2023-02-12 14:54:51,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly__g01r82tnvqn88um75mdircnc0tph0im836i805622kelvqlos99be6u2jf2k437s/family1/f775ca90473b495b85a909c1de615bc0 2023-02-12 14:54:51,283 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:51,284 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:54:51,285 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120a7f0c01b3ed04d1ea419403651defa33 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151120a7f0c01b3ed04d1ea419403651defa33 2023-02-12 14:54:51,286 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128902810d65da04a68b14b4612ea5f54e1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/f3267a7ec6e5c29f621bd6e805e83077/family1/d41d8cd98f00b204e9800998ecf8427e20151128902810d65da04a68b14b4612ea5f54e1 2023-02-12 14:54:51,286 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2 expired mob files are deleted 2023-02-12 14:54:51,286 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-12 14:54:51,287 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:51,287 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:51,287 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:51,287 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:51,287 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:51,288 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=48, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:51,289 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=48, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 892 msec 2023-02-12 14:54:51,345 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-12 14:54:51,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:51,756 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-12 14:54:51,758 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000002 2023-02-12 14:54:51,758 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/.tabledesc/.tableinfo.0000000003 2023-02-12 14:54:51,760 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly}] 2023-02-12 14:54:51,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE}] 2023-02-12 14:54:51,764 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE 2023-02-12 14:54:51,765 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:51,765 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213691765"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213691765"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213691765"}]},"ts":"1676213691765"} 2023-02-12 14:54:51,766 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=52, ppid=51, state=RUNNABLE; CloseRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:51,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:51,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aac84b1ca5b7fcfec0ef433f2e9b7077, disabling compactions & flushes 2023-02-12 14:54:51,918 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:51,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:51,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. after waiting 0 ms 2023-02-12 14:54:51,918 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:51,924 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-12 14:54:51,924 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:51,924 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aac84b1ca5b7fcfec0ef433f2e9b7077: Waiting for close lock at 1676213691918Running coprocessor pre-close hooks at 1676213691918Disabling compacts and flushes for region at 1676213691918Disabling writes for close at 1676213691918Writing region close event to WAL at 1676213691921 (+3 ms)Running coprocessor post-close hooks at 1676213691924 (+3 ms)Closed at 1676213691924 2023-02-12 14:54:51,925 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: aac84b1ca5b7fcfec0ef433f2e9b7077 to self. 2023-02-12 14:54:51,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:51,926 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=CLOSED 2023-02-12 14:54:51,926 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213691926"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213691926"}]},"ts":"1676213691926"} 2023-02-12 14:54:51,929 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=52, resume processing ppid=51 2023-02-12 14:54:51,929 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=52, ppid=51, state=SUCCESS; CloseRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001 in 161 msec 2023-02-12 14:54:51,929 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=51, ppid=50, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=true 2023-02-12 14:54:52,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:52,080 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213692079"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213692079"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213692079"}]},"ts":"1676213692079"} 2023-02-12 14:54:52,081 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=53, ppid=51, state=RUNNABLE; OpenRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:52,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:52,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => aac84b1ca5b7fcfec0ef433f2e9b7077, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:54:52,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:52,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:52,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:52,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:52,237 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:52,238 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:52,238 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1 2023-02-12 14:54:52,238 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region aac84b1ca5b7fcfec0ef433f2e9b7077 columnFamilyName family1 2023-02-12 14:54:52,243 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/515146fdc6304149b3feba42ae94eea0 2023-02-12 14:54:52,246 DEBUG [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/family1/fd8beb62343d4803babfa95006912d89_SeqId_53_ 2023-02-12 14:54:52,246 INFO [StoreOpener-aac84b1ca5b7fcfec0ef433f2e9b7077-1] regionserver.HStore(310): Store=aac84b1ca5b7fcfec0ef433f2e9b7077/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:52,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:52,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:52,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:54:52,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened aac84b1ca5b7fcfec0ef433f2e9b7077; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66270167, jitterRate=-0.012497559189796448}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:54:52,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for aac84b1ca5b7fcfec0ef433f2e9b7077: Running coprocessor pre-open hook at 1676213692236Writing region info on filesystem at 1676213692236Initializing all the Stores at 1676213692237 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676213692237Cleaning up temporary data from old regions at 1676213692249 (+12 ms)Cleaning up detritus from prior splits at 1676213692250 (+1 ms)Running coprocessor post-open hooks at 1676213692253 (+3 ms)Region opened successfully at 1676213692253 2023-02-12 14:54:52,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077., pid=53, masterSystemTime=1676213692233 2023-02-12 14:54:52,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:52,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:54:52,255 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=51 updating hbase:meta row=aac84b1ca5b7fcfec0ef433f2e9b7077, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:52,256 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077.","families":{"info":[{"qualifier":"regioninfo","vlen":96,"tag":[],"timestamp":"1676213692255"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213692255"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213692255"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213692255"}]},"ts":"1676213692255"} 2023-02-12 14:54:52,258 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=53, resume processing ppid=51 2023-02-12 14:54:52,258 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=53, ppid=51, state=SUCCESS; OpenRegionProcedure aac84b1ca5b7fcfec0ef433f2e9b7077, server=jenkins-hbase4.apache.org,45319,1676213617001 in 176 msec 2023-02-12 14:54:52,259 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-12 14:54:52,259 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly, region=aac84b1ca5b7fcfec0ef433f2e9b7077, REOPEN/MOVE in 495 msec 2023-02-12 14:54:52,261 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-12 14:54:52,261 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 500 msec 2023-02-12 14:54:52,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly in 917 msec 2023-02-12 14:54:52,264 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-12 14:54:52,265 INFO [Listener at localhost/37945] compactions.TestMobCompactor(278): alter status finished 2023-02-12 14:54:52,266 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly 2023-02-12 14:54:52,267 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:52,268 DEBUG [PEWorker-4] locking.LockProcedure(309): LOCKED pid=54, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:52,269 INFO [PEWorker-4] procedure2.TimeoutExecutorThread(81): ADDED pid=54, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214292269 2023-02-12 14:54:52,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:52,269 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:54:52,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column 'family1' 2023-02-12 14:54:52,270 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:52,270 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x4cff90d4 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:54:52,274 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@28f7300c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:54:52,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,274 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,275 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' and column='family1' 2023-02-12 14:54:52,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' column='family1' 2023-02-12 14:54:52,275 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:54:52,275 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=54, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE 2023-02-12 14:54:52,276 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=54, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly.mobLock, type=EXCLUSIVE in 8 msec 2023-02-12 14:54:52,335 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly Thread=242 (was 257), OpenFileDescriptor=610 (was 631), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=192 (was 245), ProcessCount=168 (was 168), AvailableMemoryMB=3883 (was 3939) 2023-02-12 14:54:52,346 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=242, OpenFileDescriptor=610, MaxFileDescriptor=60000, SystemLoadAverage=192, ProcessCount=168, AvailableMemoryMB=3882 2023-02-12 14:54:52,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} 2023-02-12 14:54:52,349 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:52,350 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:54:52,350 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily" procId is: 55 2023-02-12 14:54:52,351 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:54:52,351 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-12 14:54:52,352 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:52,353 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 empty. 2023-02-12 14:54:52,353 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:52,353 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily regions 2023-02-12 14:54:52,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-12 14:54:52,653 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-12 14:54:52,761 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-12 14:54:52,762 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(7671): creating {ENCODED => 8aa6e64a63f09f579f8ce0eebfb86e36, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:54:52,954 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-12 14:54:53,168 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:53,168 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1603): Closing 8aa6e64a63f09f579f8ce0eebfb86e36, disabling compactions & flushes 2023-02-12 14:54:53,168 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:53,168 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:53,168 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. after waiting 0 ms 2023-02-12 14:54:53,169 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:53,169 INFO [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:53,169 DEBUG [RegionOpenAndInit-testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily-pool-0] regionserver.HRegion(1557): Region close journal for 8aa6e64a63f09f579f8ce0eebfb86e36: Waiting for close lock at 1676213693168Disabling compacts and flushes for region at 1676213693168Disabling writes for close at 1676213693169 (+1 ms)Writing region close event to WAL at 1676213693169Closed at 1676213693169 2023-02-12 14:54:53,172 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:54:53,173 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213693173"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213693173"}]},"ts":"1676213693173"} 2023-02-12 14:54:53,174 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:54:53,175 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:54:53,175 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213693175"}]},"ts":"1676213693175"} 2023-02-12 14:54:53,176 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLING in hbase:meta 2023-02-12 14:54:53,179 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, ASSIGN}] 2023-02-12 14:54:53,180 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, ASSIGN 2023-02-12 14:54:53,180 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=56, ppid=55, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:54:53,331 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:53,332 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213693331"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213693331"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213693331"}]},"ts":"1676213693331"} 2023-02-12 14:54:53,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE; OpenRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:54:53,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-12 14:54:53,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:53,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8aa6e64a63f09f579f8ce0eebfb86e36, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:54:53,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:53,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:54:53,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:53,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:53,491 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:53,492 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:54:53,492 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:54:53,493 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8aa6e64a63f09f579f8ce0eebfb86e36 columnFamilyName family1 2023-02-12 14:54:53,494 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(310): Store=8aa6e64a63f09f579f8ce0eebfb86e36/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:54:53,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:53,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:53,498 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:54:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:54:53,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8aa6e64a63f09f579f8ce0eebfb86e36; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73054874, jitterRate=0.0886024534702301}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:54:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8aa6e64a63f09f579f8ce0eebfb86e36: Running coprocessor pre-open hook at 1676213693489Writing region info on filesystem at 1676213693489Initializing all the Stores at 1676213693491 (+2 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} at 1676213693491Cleaning up temporary data from old regions at 1676213693496 (+5 ms)Cleaning up detritus from prior splits at 1676213693496Running coprocessor post-open hooks at 1676213693500 (+4 ms)Region opened successfully at 1676213693500 2023-02-12 14:54:53,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., pid=57, masterSystemTime=1676213693485 2023-02-12 14:54:53,502 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:53,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:53,502 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=56 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:53,502 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213693502"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213693502"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213693502"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213693502"}]},"ts":"1676213693502"} 2023-02-12 14:54:53,504 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-12 14:54:53,505 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; OpenRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001 in 170 msec 2023-02-12 14:54:53,506 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=56, resume processing ppid=55 2023-02-12 14:54:53,506 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=56, ppid=55, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, ASSIGN in 326 msec 2023-02-12 14:54:53,506 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:54:53,507 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213693506"}]},"ts":"1676213693506"} 2023-02-12 14:54:53,507 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, state=ENABLED in hbase:meta 2023-02-12 14:54:53,511 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=55, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:54:53,512 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=55, state=SUCCESS; CreateTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 1.1630 sec 2023-02-12 14:54:54,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=55 2023-02-12 14:54:54,457 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, procId: 55 completed 2023-02-12 14:54:54,474 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:54,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,476 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:54,476 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:54,476 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:54:54,476 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:54:54,477 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,477 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,478 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:54,478 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:54,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:54,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:54,478 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:54,478 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,479 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:54,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,479 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,479 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:54,479 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:54:54,480 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:54,480 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:54:54,481 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:54,481 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:54,481 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:54,481 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:54:54,482 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-12 14:54:54,902 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130d0b0166cd2494597a2f196fb1fca3258 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130d0b0166cd2494597a2f196fb1fca3258 2023-02-12 14:54:55,306 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/33cc64fc656e480294dd0ac2b67b3cb5 2023-02-12 14:54:55,310 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/33cc64fc656e480294dd0ac2b67b3cb5 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/33cc64fc656e480294dd0ac2b67b3cb5 2023-02-12 14:54:55,314 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/33cc64fc656e480294dd0ac2b67b3cb5, entries=1000, sequenceid=5, filesize=190.5 K 2023-02-12 14:54:55,315 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 833ms, sequenceid=5, compaction requested=false 2023-02-12 14:54:55,315 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:55,315 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:54:55,315 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:54:55,316 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:54:55,317 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,317 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:55,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:55,317 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,317 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:55,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:54:55,319 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@60e5feaf[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:55,319 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:54:55,319 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,320 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:54:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:54:55,320 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:55,320 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,320 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:55,322 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:55,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:55,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:55,323 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:55,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:54:55,325 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:54:55,325 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:55,325 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:55,325 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,325 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:55,326 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,326 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:55,326 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:55,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:55,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:55,327 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:55,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,328 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:55,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:55,334 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:55,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:55,334 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,334 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:55,335 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:55,435 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:55,436 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:55,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:55,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:55,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,441 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:55,441 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:55,441 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:54:55,441 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:54:55,442 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,442 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,444 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,444 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:55,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:55,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:55,444 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:55,444 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:55,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,444 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:55,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,445 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:54:55,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:55,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:54:55,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:55,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:54:55,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:55,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:55,445 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:55,445 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:54:55,445 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-12 14:54:55,586 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:54:55,857 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015112839e6a205491a4af98ea14c845915d751 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112839e6a205491a4af98ea14c845915d751 2023-02-12 14:54:56,129 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly' 2023-02-12 14:54:56,263 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/144aa5ebb0394c3c995c845aff8f9adf 2023-02-12 14:54:56,267 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/144aa5ebb0394c3c995c845aff8f9adf as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/144aa5ebb0394c3c995c845aff8f9adf 2023-02-12 14:54:56,271 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/144aa5ebb0394c3c995c845aff8f9adf, entries=1, sequenceid=9, filesize=5.0 K 2023-02-12 14:54:56,272 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 827ms, sequenceid=9, compaction requested=false 2023-02-12 14:54:56,272 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:54:56,272 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:54:56,272 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:54:56,273 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,273 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:56,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:56,273 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,273 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:56,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:54:56,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58319432[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:56,275 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:54:56,275 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,275 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,276 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,276 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:54:56,276 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:54:56,276 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:56,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:56,277 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:54:56,277 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:56,277 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,277 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:56,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:56,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:54:56,279 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:54:56,279 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:56,279 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:56,279 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,279 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:56,281 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,281 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:56,281 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:56,282 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,283 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:56,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:56,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-12 14:54:56,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:56,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:56,285 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:56,285 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:56,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:56,386 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:56,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:56,390 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:56,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:56,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,392 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:56,392 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:56,392 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:54:56,392 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:54:56,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,394 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,394 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:56,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:56,394 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:56,394 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,395 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,395 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:54:56,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:56,395 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:56,395 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:56,395 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:54:56,396 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:56,808 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512016e53e4b4c0c241deb53622a2c1c914cc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512016e53e4b4c0c241deb53622a2c1c914cc 2023-02-12 14:54:57,213 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/ccc8b297344845c984a7cf90dd446a55 2023-02-12 14:54:57,218 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/ccc8b297344845c984a7cf90dd446a55 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/ccc8b297344845c984a7cf90dd446a55 2023-02-12 14:54:57,221 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/ccc8b297344845c984a7cf90dd446a55, entries=1, sequenceid=13, filesize=5.0 K 2023-02-12 14:54:57,222 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 827ms, sequenceid=13, compaction requested=true 2023-02-12 14:54:57,222 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:54:57,222 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:54:57,222 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:54:57,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,224 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:57,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:57,224 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,224 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:57,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:57,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:57,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:57,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:54:57,226 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47c77b6c[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:57,226 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:54:57,226 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,228 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:54:57,228 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:54:57,228 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:57,228 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,228 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:57,229 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:54:57,229 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:57,229 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,229 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:54:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:57,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:57,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:54:57,232 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:54:57,232 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:57,232 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:57,232 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,232 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:57,233 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,233 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:57,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:57,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,233 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:57,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:57,234 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,235 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:57,237 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:57,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:57,238 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:57,238 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-12 14:54:57,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:57,238 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:57,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:57,238 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,338 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:57,339 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:57,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:57,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:57,344 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,344 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:57,344 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:57,344 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:54:57,344 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:54:57,345 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,345 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,347 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,347 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:57,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:57,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:57,347 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:57,347 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:57,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,347 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:57,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,348 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,348 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:54:57,348 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:57,348 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:54:57,348 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:57,348 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:54:57,348 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:57,348 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:57,348 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:57,348 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:54:57,348 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:57,762 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015120567ab90362e7048cd8fc90296ea4a4526 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120567ab90362e7048cd8fc90296ea4a4526 2023-02-12 14:54:58,167 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/cade35c5cb6d456aa48e6b8a0243f4c2 2023-02-12 14:54:58,172 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/cade35c5cb6d456aa48e6b8a0243f4c2 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/cade35c5cb6d456aa48e6b8a0243f4c2 2023-02-12 14:54:58,176 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/cade35c5cb6d456aa48e6b8a0243f4c2, entries=1, sequenceid=17, filesize=5.0 K 2023-02-12 14:54:58,176 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 828ms, sequenceid=17, compaction requested=true 2023-02-12 14:54:58,176 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:54:58,176 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:58,176 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:58,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:58,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:54:58,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:54:58,178 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,178 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:58,178 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,178 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:58,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:58,179 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:54:58,180 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7806acd[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:58,180 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:54:58,180 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,184 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,184 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:54:58,184 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:54:58,184 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:58,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,184 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:58,186 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:54:58,186 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:58,186 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,186 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:54:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:58,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:58,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:54:58,188 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:54:58,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:58,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:58,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,188 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:58,189 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,189 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,189 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:58,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:58,189 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:58,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:58,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:58,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:58,190 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:58,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,191 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:58,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,193 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:58,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:58,193 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,193 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:58,194 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:58,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:58,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:58,294 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:58,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:58,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:58,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:58,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,300 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:58,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:58,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:54:58,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:54:58,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,300 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,301 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,301 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:58,301 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:58,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:58,302 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:58,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,302 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:58,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:58,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,302 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:54:58,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:58,303 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:58,303 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:58,303 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:54:58,303 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:58,715 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288cb1a4e3d8304dfaa1064bd3c2a9d7e8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512288cb1a4e3d8304dfaa1064bd3c2a9d7e8 2023-02-12 14:54:59,120 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/3c9ee24e4b1d456eb4210349acfc5527 2023-02-12 14:54:59,125 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/3c9ee24e4b1d456eb4210349acfc5527 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/3c9ee24e4b1d456eb4210349acfc5527 2023-02-12 14:54:59,128 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/3c9ee24e4b1d456eb4210349acfc5527, entries=1, sequenceid=21, filesize=5.0 K 2023-02-12 14:54:59,129 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 826ms, sequenceid=21, compaction requested=true 2023-02-12 14:54:59,129 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:54:59,129 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:54:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:54:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:54:59,129 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:54:59,135 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,135 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:59,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:59,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:54:59,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48dfec14[Count = 0] remaining members to acquire global barrier 2023-02-12 14:54:59,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:54:59,136 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,137 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:54:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:54:59,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:54:59,138 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,138 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:54:59,139 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:54:59,139 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,139 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:59,139 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:54:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:59,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:59,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:54:59,141 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:54:59,141 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:54:59,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:54:59,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:54:59,141 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,141 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:54:59,142 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,142 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:54:59,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:54:59,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,142 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:59,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:54:59,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:59,143 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:59,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:54:59,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,144 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:54:59,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,147 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:54:59,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:59,147 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:54:59,147 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:54:59,248 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:54:59,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:54:59,252 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:54:59,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:54:59,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,254 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:54:59,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:59,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:54:59,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:54:59,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,254 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,255 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,255 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:54:59,255 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:54:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:54:59,256 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:54:59,256 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:54:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,256 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:54:59,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,256 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,257 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:54:59,257 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:54:59,257 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:54:59,257 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:54:59,257 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:54:59,257 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:54:59,257 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:54:59,257 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:54:59,257 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:54:59,257 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:54:59,668 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123116ecc24b4f544309a65bdd500b770b01 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123116ecc24b4f544309a65bdd500b770b01 2023-02-12 14:55:00,074 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/e577fcd83f9f4e95bc639a5657997743 2023-02-12 14:55:00,079 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/e577fcd83f9f4e95bc639a5657997743 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e577fcd83f9f4e95bc639a5657997743 2023-02-12 14:55:00,082 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e577fcd83f9f4e95bc639a5657997743, entries=1, sequenceid=25, filesize=5.0 K 2023-02-12 14:55:00,083 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 826ms, sequenceid=25, compaction requested=true 2023-02-12 14:55:00,083 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:00,083 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:55:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:55:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:55:00,083 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:55:00,085 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,085 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:00,085 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,085 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:00,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:00,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:55:00,086 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1c1b6139[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:00,086 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:55:00,086 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,087 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,087 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:55:00,087 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:55:00,087 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,087 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:00,087 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:00,089 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,089 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:55:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:00,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:00,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:55:00,091 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:55:00,091 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:00,091 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:00,091 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,092 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:00,093 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,093 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,093 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:00,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:00,093 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:00,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:00,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:00,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,095 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:00,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,098 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,098 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:00,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:00,099 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:00,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:00,099 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-12 14:55:00,099 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:00,099 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:00,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:00,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:00,199 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:00,200 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:00,204 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:00,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:00,205 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,205 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:00,205 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:00,206 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:55:00,206 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:55:00,206 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,206 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,211 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:00,211 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:00,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:00,212 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:00,212 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:00,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,212 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:00,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,212 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,212 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:55:00,212 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:00,213 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:55:00,213 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:00,213 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:55:00,213 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:55:00,213 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:00,213 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:00,213 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:55:00,213 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:55:00,625 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2016010156a12c3a81eb458cad6fa9fdbab8e2b3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010156a12c3a81eb458cad6fa9fdbab8e2b3 2023-02-12 14:55:01,030 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/1b5f5428938b406aa49239fc280f7997 2023-02-12 14:55:01,036 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/1b5f5428938b406aa49239fc280f7997 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1b5f5428938b406aa49239fc280f7997 2023-02-12 14:55:01,039 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1b5f5428938b406aa49239fc280f7997, entries=1, sequenceid=29, filesize=5.0 K 2023-02-12 14:55:01,040 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 827ms, sequenceid=29, compaction requested=true 2023-02-12 14:55:01,040 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:01,040 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:01,040 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:55:01,040 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:55:01,040 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,040 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:55:01,040 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:55:01,043 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,043 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:01,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:01,044 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,044 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:01,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:55:01,045 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@607b75cd[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:01,045 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:55:01,045 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,046 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:55:01,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:55:01,046 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,046 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:01,046 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:01,047 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:55:01,047 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,047 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:01,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:01,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:01,048 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:55:01,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:01,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:01,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:01,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:01,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:55:01,050 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:55:01,050 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:01,050 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:01,050 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,050 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:01,051 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,051 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:01,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:01,051 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:01,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:01,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:01,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:01,052 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,053 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,056 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:01,056 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-12 14:55:01,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:01,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:01,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:01,056 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,056 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:01,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:01,057 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:01,057 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:01,157 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:01,158 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:01,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:01,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:01,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,163 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:01,164 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:01,164 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:55:01,164 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:55:01,164 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,164 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,165 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:01,165 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:01,166 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,166 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,167 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:55:01,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,167 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:55:01,167 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:01,167 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:55:01,167 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:55:01,167 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:01,168 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:01,168 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:55:01,168 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:55:01,186 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103ff8eeaf6b9c44b6588b6687da0e9de89 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103ff8eeaf6b9c44b6588b6687da0e9de89 2023-02-12 14:55:01,592 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/1ba1c042b32142848ef9a6dd87e33c1d 2023-02-12 14:55:01,597 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/1ba1c042b32142848ef9a6dd87e33c1d as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1ba1c042b32142848ef9a6dd87e33c1d 2023-02-12 14:55:01,601 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1ba1c042b32142848ef9a6dd87e33c1d, entries=1, sequenceid=33, filesize=5.0 K 2023-02-12 14:55:01,602 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 434ms, sequenceid=33, compaction requested=true 2023-02-12 14:55:01,602 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:01,602 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:01,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:55:01,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:55:01,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:55:01,602 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:55:01,603 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,603 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:01,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:01,604 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,604 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:01,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:55:01,605 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6429bd0d[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:01,605 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:55:01,605 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,606 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,606 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:55:01,606 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:55:01,606 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:01,606 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,606 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:01,608 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:55:01,608 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,609 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:01,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:01,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:01,609 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:55:01,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:01,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:01,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:01,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:01,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:55:01,611 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:55:01,611 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:01,611 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:01,611 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,611 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:01,612 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,612 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,612 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:01,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:01,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:01,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,613 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:01,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:01,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:01,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,614 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:01,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:01,616 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:01,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:01,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:01,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:01,617 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:01,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:01,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:01,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,617 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,617 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:01,617 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:01,717 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:01,718 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:01,722 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:01,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:01,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,723 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:01,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:01,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:55:01,723 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:55:01,724 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,724 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,726 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,726 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:01,726 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:01,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:01,726 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:01,726 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:01,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:01,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,727 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:55:01,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:01,727 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:55:01,727 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:01,727 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:55:01,727 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:55:01,727 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:01,727 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:01,728 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:55:01,728 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:55:01,739 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201509072bde40c32a3c4ae8af967f0eee934405 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201509072bde40c32a3c4ae8af967f0eee934405 2023-02-12 14:55:02,144 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/1bdaa92be1fe4db5be5e5948332f1423 2023-02-12 14:55:02,149 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/1bdaa92be1fe4db5be5e5948332f1423 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1bdaa92be1fe4db5be5e5948332f1423 2023-02-12 14:55:02,153 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1bdaa92be1fe4db5be5e5948332f1423, entries=1, sequenceid=37, filesize=5.0 K 2023-02-12 14:55:02,154 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 426ms, sequenceid=37, compaction requested=true 2023-02-12 14:55:02,154 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:02,154 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:02,154 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:55:02,154 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:55:02,154 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,155 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:55:02,155 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:55:02,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:02,156 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:02,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:55:02,158 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@53a142d0[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:02,158 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:55:02,158 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,159 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:55:02,159 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:55:02,159 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:02,159 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:02,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:55:02,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:02,160 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,160 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:02,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:02,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:02,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:02,163 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:55:02,163 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:55:02,163 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:02,163 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:02,163 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,163 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:02,166 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,166 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,166 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:02,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:02,166 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:02,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:02,167 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:02,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,169 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,169 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,169 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:02,169 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,169 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:02,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:02,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:02,169 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:02,169 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:02,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:02,170 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,170 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:02,170 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-12 14:55:02,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:02,170 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,170 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:02,170 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:02,270 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:02,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:02,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:02,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:02,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,276 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:02,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:02,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' 2023-02-12 14:55:02,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', kicking off acquire phase on members. 2023-02-12 14:55:02,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,277 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,278 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:02,278 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:02,278 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:02,278 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:02,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,278 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:02,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,279 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,279 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily is still present, removing 2023-02-12 14:55:02,279 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:02,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' with timeout 60000ms 2023-02-12 14:55:02,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:02,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'acquire' stage 2023-02-12 14:55:02,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:55:02,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:02,279 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:02,279 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. started... 2023-02-12 14:55:02,279 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 8aa6e64a63f09f579f8ce0eebfb86e36 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:55:02,691 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511207f041d6b5cdc414f9e9b28b2e203418d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511207f041d6b5cdc414f9e9b28b2e203418d 2023-02-12 14:55:03,096 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/201251153f384b6ba807604ede1c985d 2023-02-12 14:55:03,101 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/201251153f384b6ba807604ede1c985d as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/201251153f384b6ba807604ede1c985d 2023-02-12 14:55:03,104 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/201251153f384b6ba807604ede1c985d, entries=1, sequenceid=41, filesize=5.0 K 2023-02-12 14:55:03,105 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 8aa6e64a63f09f579f8ce0eebfb86e36 in 826ms, sequenceid=41, compaction requested=true 2023-02-12 14:55:03,105 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:03,105 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally acquired 2023-02-12 14:55:03,105 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily) in zk 2023-02-12 14:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,107 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:03,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,107 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:03,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:03,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:03,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:03,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' on coordinator 2023-02-12 14:55:03,109 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14f4f334[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:03,109 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' starting 'in-barrier' execution. 2023-02-12 14:55:03,109 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,111 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,111 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' received 'reached' from coordinator. 2023-02-12 14:55:03,111 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' locally completed 2023-02-12 14:55:03,111 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:03,111 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,111 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:03,112 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' has notified controller of completion 2023-02-12 14:55:03,112 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:03,112 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,112 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' completed. 2023-02-12 14:55:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:03,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:03,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:03,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', counting down latch. Waiting for 0 more 2023-02-12 14:55:03,114 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' execution completed 2023-02-12 14:55:03,114 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:03,114 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:03,114 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,114 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDailyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:03,116 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,116 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:03,116 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:03,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:03,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:03,116 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,117 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:03,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:03,119 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily because node does not exist (not an error) 2023-02-12 14:55:03,119 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:03,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:03,119 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:03,119 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:03,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:03,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:03,120 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:03,120 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:03,220 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:03,221 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:03,222 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:03,223 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:03,224 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=58, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:03,225 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:03,225 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=58, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214303225 2023-02-12 14:55:03,227 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:55:03,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-12 14:55:03,228 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:03,229 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x047264c2 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:03,234 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2e43cde5, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:03,234 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,072 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601031b8f194e3a804f3c82d3d2eb4052b8a0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201601031b8f194e3a804f3c82d3d2eb4052b8a0 2023-02-12 14:55:04,073 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:04,085 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:04,086 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42162, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:04,092 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:55:04,093 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40510, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:55:04,101 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d6d8c48d92fd458192809acc762b6c69 first=Optional[r3] last=Optional[r6] 2023-02-12 14:55:04,101 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d6d8c48d92fd458192809acc762b6c69}] 2023-02-12 14:55:04,105 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d6d8c48d92fd458192809acc762b6c69 for inclusion in 8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:04,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r3 last=r6 2023-02-12 14:55:04,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:55:04,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:04,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228/family1/d6d8c48d92fd458192809acc762b6c69 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rch5c6l1n3d8a278r5orjhmk7ttvj2cbq6e9c1ii55o423rtf6ong70tp70e2kl9/family1/d6d8c48d92fd458192809acc762b6c69 2023-02-12 14:55:04,110 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rch5c6l1n3d8a278r5orjhmk7ttvj2cbq6e9c1ii55o423rtf6ong70tp70e2kl9/family1/d6d8c48d92fd458192809acc762b6c69 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_ 2023-02-12 14:55:04,111 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rch5c6l1n3d8a278r5orjhmk7ttvj2cbq6e9c1ii55o423rtf6ong70tp70e2kl9/family1/d6d8c48d92fd458192809acc762b6c69 into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_ - updating store file list. 2023-02-12 14:55:04,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_ into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:04,113 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rch5c6l1n3d8a278r5orjhmk7ttvj2cbq6e9c1ii55o423rtf6ong70tp70e2kl9/family1/d6d8c48d92fd458192809acc762b6c69 into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_) 2023-02-12 14:55:04,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__rch5c6l1n3d8a278r5orjhmk7ttvj2cbq6e9c1ii55o423rtf6ong70tp70e2kl9/family1/d6d8c48d92fd458192809acc762b6c69 2023-02-12 14:55:04,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-12 14:55:04,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 51552 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-12 14:55:04,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 8aa6e64a63f09f579f8ce0eebfb86e36/family1 is initiating minor compaction 2023-02-12 14:55:04,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@58f15fd8; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:04,116 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8aa6e64a63f09f579f8ce0eebfb86e36/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:04,117 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/144aa5ebb0394c3c995c845aff8f9adf, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/ccc8b297344845c984a7cf90dd446a55, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/cade35c5cb6d456aa48e6b8a0243f4c2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/3c9ee24e4b1d456eb4210349acfc5527, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e577fcd83f9f4e95bc639a5657997743, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1b5f5428938b406aa49239fc280f7997, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1ba1c042b32142848ef9a6dd87e33c1d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1bdaa92be1fe4db5be5e5948332f1423, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/201251153f384b6ba807604ede1c985d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp, totalSize=50.3 K 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 144aa5ebb0394c3c995c845aff8f9adf, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting ccc8b297344845c984a7cf90dd446a55, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting cade35c5cb6d456aa48e6b8a0243f4c2, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 3c9ee24e4b1d456eb4210349acfc5527, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting e577fcd83f9f4e95bc639a5657997743, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 1b5f5428938b406aa49239fc280f7997, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 1ba1c042b32142848ef9a6dd87e33c1d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 1bdaa92be1fe4db5be5e5948332f1423, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 201251153f384b6ba807604ede1c985d, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-12 14:55:04,117 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_, keycount=4, bloomtype=ROW, size=5.6 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-12 14:55:04,118 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151228 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:04,119 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:04,120 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512288cb1a4e3d8304dfaa1064bd3c2a9d7e8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512288cb1a4e3d8304dfaa1064bd3c2a9d7e8 2023-02-12 14:55:04,121 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123116ecc24b4f544309a65bdd500b770b01 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015123116ecc24b4f544309a65bdd500b770b01 2023-02-12 14:55:04,122 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010156a12c3a81eb458cad6fa9fdbab8e2b3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2016010156a12c3a81eb458cad6fa9fdbab8e2b3 2023-02-12 14:55:04,123 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103ff8eeaf6b9c44b6588b6687da0e9de89 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20160103ff8eeaf6b9c44b6588b6687da0e9de89 2023-02-12 14:55:04,123 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 4 expired mob files are deleted 2023-02-12 14:55:04,123 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151228' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151228 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151116 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151123 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,124 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:04,148 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8aa6e64a63f09f579f8ce0eebfb86e36#family1#compaction#72 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:04,577 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/baf1e8f77ff546f0a0acad1e50a44fdf as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/baf1e8f77ff546f0a0acad1e50a44fdf 2023-02-12 14:55:04,581 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 8aa6e64a63f09f579f8ce0eebfb86e36/family1 of 8aa6e64a63f09f579f8ce0eebfb86e36 into baf1e8f77ff546f0a0acad1e50a44fdf(size=6.8 K), total size for store is 197.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:04,581 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:04,581 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., storeName=family1, fileCount=10, fileSize=50.3 K (5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.0 K, 5.6 K), priority=2, time=1676213704116; duration=0sec 2023-02-12 14:55:04,582 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:04,662 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/144aa5ebb0394c3c995c845aff8f9adf, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/ccc8b297344845c984a7cf90dd446a55, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/cade35c5cb6d456aa48e6b8a0243f4c2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/3c9ee24e4b1d456eb4210349acfc5527, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e577fcd83f9f4e95bc639a5657997743, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1b5f5428938b406aa49239fc280f7997, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1ba1c042b32142848ef9a6dd87e33c1d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1bdaa92be1fe4db5be5e5948332f1423, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/201251153f384b6ba807604ede1c985d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_] to archive 2023-02-12 14:55:04,662 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:04,664 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/144aa5ebb0394c3c995c845aff8f9adf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/144aa5ebb0394c3c995c845aff8f9adf 2023-02-12 14:55:04,665 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/ccc8b297344845c984a7cf90dd446a55 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/ccc8b297344845c984a7cf90dd446a55 2023-02-12 14:55:04,666 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/cade35c5cb6d456aa48e6b8a0243f4c2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/cade35c5cb6d456aa48e6b8a0243f4c2 2023-02-12 14:55:04,667 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/3c9ee24e4b1d456eb4210349acfc5527 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/3c9ee24e4b1d456eb4210349acfc5527 2023-02-12 14:55:04,668 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e577fcd83f9f4e95bc639a5657997743 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e577fcd83f9f4e95bc639a5657997743 2023-02-12 14:55:04,669 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1b5f5428938b406aa49239fc280f7997 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1b5f5428938b406aa49239fc280f7997 2023-02-12 14:55:04,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1ba1c042b32142848ef9a6dd87e33c1d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1ba1c042b32142848ef9a6dd87e33c1d 2023-02-12 14:55:04,671 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1bdaa92be1fe4db5be5e5948332f1423 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/1bdaa92be1fe4db5be5e5948332f1423 2023-02-12 14:55:04,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/201251153f384b6ba807604ede1c985d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/201251153f384b6ba807604ede1c985d 2023-02-12 14:55:04,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/fed33829a3ab4917a6b9e5fb34725e04_SeqId_44_ 2023-02-12 14:55:04,979 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205aa434fd511d54a74b575bd574804e6a6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151205aa434fd511d54a74b575bd574804e6a6 2023-02-12 14:55:04,980 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:05,004 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1b8b9ec165194d65b13c740b15c9137d first=Optional[r00] last=Optional[r2] 2023-02-12 14:55:05,005 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1b8b9ec165194d65b13c740b15c9137d}] 2023-02-12 14:55:05,007 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1b8b9ec165194d65b13c740b15c9137d for inclusion in 8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:05,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r00 last=r2 2023-02-12 14:55:05,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:55:05,011 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:05,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130/family1/1b8b9ec165194d65b13c740b15c9137d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fimj9sg90s7346n82e86guqroi43leectv66b716omesfu1hlafcs8hsckrelmtu/family1/1b8b9ec165194d65b13c740b15c9137d 2023-02-12 14:55:05,014 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fimj9sg90s7346n82e86guqroi43leectv66b716omesfu1hlafcs8hsckrelmtu/family1/1b8b9ec165194d65b13c740b15c9137d as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_ 2023-02-12 14:55:05,014 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fimj9sg90s7346n82e86guqroi43leectv66b716omesfu1hlafcs8hsckrelmtu/family1/1b8b9ec165194d65b13c740b15c9137d into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_ - updating store file list. 2023-02-12 14:55:05,017 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_ into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:05,018 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fimj9sg90s7346n82e86guqroi43leectv66b716omesfu1hlafcs8hsckrelmtu/family1/1b8b9ec165194d65b13c740b15c9137d into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_) 2023-02-12 14:55:05,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__fimj9sg90s7346n82e86guqroi43leectv66b716omesfu1hlafcs8hsckrelmtu/family1/1b8b9ec165194d65b13c740b15c9137d 2023-02-12 14:55:05,018 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 0 compacting, 3 eligible, 16 blocking 2023-02-12 14:55:05,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 3 files of size 397422 starting at candidate #0 after considering 1 permutations with 1 in ratio 2023-02-12 14:55:05,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 8aa6e64a63f09f579f8ce0eebfb86e36/family1 is initiating minor compaction (all files) 2023-02-12 14:55:05,019 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1520433; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:05,019 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 8aa6e64a63f09f579f8ce0eebfb86e36/family1 in testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:05,019 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/33cc64fc656e480294dd0ac2b67b3cb5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/baf1e8f77ff546f0a0acad1e50a44fdf, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp, totalSize=388.1 K 2023-02-12 14:55:05,020 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 33cc64fc656e480294dd0ac2b67b3cb5, keycount=1000, bloomtype=ROW, size=190.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840591 2023-02-12 14:55:05,020 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting baf1e8f77ff546f0a0acad1e50a44fdf, keycount=9, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240591 2023-02-12 14:55:05,020 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_, keycount=1002, bloomtype=ROW, size=190.9 K, encoding=NONE, compression=NONE, seqNum=48, earliestPutTs=1448878840591 2023-02-12 14:55:05,021 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151130 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:05,021 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:05,023 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130d0b0166cd2494597a2f196fb1fca3258 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e20151130d0b0166cd2494597a2f196fb1fca3258 2023-02-12 14:55:05,024 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512016e53e4b4c0c241deb53622a2c1c914cc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201512016e53e4b4c0c241deb53622a2c1c914cc 2023-02-12 14:55:05,026 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120567ab90362e7048cd8fc90296ea4a4526 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015120567ab90362e7048cd8fc90296ea4a4526 2023-02-12 14:55:05,026 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 3 expired mob files are deleted 2023-02-12 14:55:05,026 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151130' 2023-02-12 14:55:05,026 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151130 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:05,026 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 5 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:05,027 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:05,028 DEBUG [PEWorker-4] locking.LockProcedure(242): UNLOCKED pid=58, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:05,029 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=58, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 1.8050 sec 2023-02-12 14:55:05,038 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8aa6e64a63f09f579f8ce0eebfb86e36#family1#compaction#73 average throughput is 22.45 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:05,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'WEEKLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-12 14:55:05,107 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=59, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:05,463 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/.tmp/family1/5958189e9f614520a838583947cd0186 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/5958189e9f614520a838583947cd0186 2023-02-12 14:55:05,468 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 3 (all) file(s) in 8aa6e64a63f09f579f8ce0eebfb86e36/family1 of 8aa6e64a63f09f579f8ce0eebfb86e36 into 5958189e9f614520a838583947cd0186(size=192.0 K), total size for store is 192.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:05,468 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:05,468 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., storeName=family1, fileCount=3, fileSize=388.1 K (190.5 K, 6.8 K, 190.9 K), priority=2, time=1676213705019; duration=0sec 2023-02-12 14:55:05,468 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:05,516 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-12 14:55:05,517 DEBUG [PEWorker-2] util.FSTableDescriptors(519): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000001 2023-02-12 14:55:05,517 INFO [PEWorker-2] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-12 14:55:05,519 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=60, ppid=59, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-12 14:55:05,521 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE}] 2023-02-12 14:55:05,522 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE 2023-02-12 14:55:05,522 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:05,522 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213705522"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213705522"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213705522"}]},"ts":"1676213705522"} 2023-02-12 14:55:05,524 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=61, state=RUNNABLE; CloseRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:05,552 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/33cc64fc656e480294dd0ac2b67b3cb5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/baf1e8f77ff546f0a0acad1e50a44fdf, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_] to archive 2023-02-12 14:55:05,553 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:05,555 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/33cc64fc656e480294dd0ac2b67b3cb5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/33cc64fc656e480294dd0ac2b67b3cb5 2023-02-12 14:55:05,556 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/baf1e8f77ff546f0a0acad1e50a44fdf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/baf1e8f77ff546f0a0acad1e50a44fdf 2023-02-12 14:55:05,557 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/e9b427ea1a6e4bdfb26a0880c67d6237_SeqId_47_ 2023-02-12 14:55:05,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:05,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8aa6e64a63f09f579f8ce0eebfb86e36, disabling compactions & flushes 2023-02-12 14:55:05,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:05,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:05,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. after waiting 0 ms 2023-02-12 14:55:05,675 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:05,680 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-12 14:55:05,681 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:05,681 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8aa6e64a63f09f579f8ce0eebfb86e36: Waiting for close lock at 1676213705675Running coprocessor pre-close hooks at 1676213705675Disabling compacts and flushes for region at 1676213705675Disabling writes for close at 1676213705675Writing region close event to WAL at 1676213705677 (+2 ms)Running coprocessor post-close hooks at 1676213705680 (+3 ms)Closed at 1676213705681 (+1 ms) 2023-02-12 14:55:05,681 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 8aa6e64a63f09f579f8ce0eebfb86e36 to self. 2023-02-12 14:55:05,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:05,683 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=CLOSED 2023-02-12 14:55:05,683 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213705682"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213705682"}]},"ts":"1676213705682"} 2023-02-12 14:55:05,685 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=61 2023-02-12 14:55:05,685 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=61, state=SUCCESS; CloseRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001 in 161 msec 2023-02-12 14:55:05,685 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=61, ppid=60, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=true 2023-02-12 14:55:05,836 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:05,836 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213705836"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213705836"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213705836"}]},"ts":"1676213705836"} 2023-02-12 14:55:05,837 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=63, ppid=61, state=RUNNABLE; OpenRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:05,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:05,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8aa6e64a63f09f579f8ce0eebfb86e36, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:55:05,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:05,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:05,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:05,992 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:05,993 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:05,994 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:05,994 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:05,994 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8aa6e64a63f09f579f8ce0eebfb86e36 columnFamilyName family1 2023-02-12 14:55:05,998 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/5958189e9f614520a838583947cd0186 2023-02-12 14:55:05,998 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(310): Store=8aa6e64a63f09f579f8ce0eebfb86e36/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:05,999 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:06,000 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:06,002 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:06,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8aa6e64a63f09f579f8ce0eebfb86e36; next sequenceid=51; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68076375, jitterRate=0.01441703736782074}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:55:06,003 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8aa6e64a63f09f579f8ce0eebfb86e36: Running coprocessor pre-open hook at 1676213705992Writing region info on filesystem at 1676213705992Initializing all the Stores at 1676213705992Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676213705993 (+1 ms)Cleaning up temporary data from old regions at 1676213706000 (+7 ms)Cleaning up detritus from prior splits at 1676213706001 (+1 ms)Running coprocessor post-open hooks at 1676213706003 (+2 ms)Region opened successfully at 1676213706003 2023-02-12 14:55:06,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., pid=63, masterSystemTime=1676213705988 2023-02-12 14:55:06,004 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:06,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:06,005 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=61 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=OPEN, openSeqNum=51, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:06,005 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213706005"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213706005"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213706005"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213706005"}]},"ts":"1676213706005"} 2023-02-12 14:55:06,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=63, resume processing ppid=61 2023-02-12 14:55:06,007 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, ppid=61, state=SUCCESS; OpenRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001 in 169 msec 2023-02-12 14:55:06,008 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=60 2023-02-12 14:55:06,008 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=60, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE in 486 msec 2023-02-12 14:55:06,010 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=59 2023-02-12 14:55:06,010 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=59, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 489 msec 2023-02-12 14:55:06,011 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=59, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 904 msec 2023-02-12 14:55:06,012 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=59 2023-02-12 14:55:06,013 INFO [Listener at localhost/37945] compactions.TestMobCompactor(278): alter status finished 2023-02-12 14:55:06,014 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:06,015 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:06,015 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=64, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:06,016 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:06,016 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=64, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214306016 2023-02-12 14:55:06,017 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:55:06,017 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 5 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-12 14:55:06,017 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:06,018 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x199904bc to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:06,022 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@b374244, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:06,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,022 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,845 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511287e5ef8f29b694ca39a4df780a1e15690 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511287e5ef8f29b694ca39a4df780a1e15690 2023-02-12 14:55:06,846 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:06,858 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:06,859 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42166, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:06,865 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:55:06,867 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40512, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:55:06,875 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f2770e92eb66401a94ef9b5748d28960 first=Optional[r05] last=Optional[r8] 2023-02-12 14:55:06,875 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=51 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f2770e92eb66401a94ef9b5748d28960}] 2023-02-12 14:55:06,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f2770e92eb66401a94ef9b5748d28960 for inclusion in 8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:06,881 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r05 last=r8 2023-02-12 14:55:06,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:55:06,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 8aa6e64a63f09f579f8ce0eebfb86e36: 2023-02-12 14:55:06,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101/family1/f2770e92eb66401a94ef9b5748d28960 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q0464m8kkuhi9m0sqio4f7e5l3q60idg7kouhdmeofo0dra3f8idfbekcval6k87/family1/f2770e92eb66401a94ef9b5748d28960 2023-02-12 14:55:06,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q0464m8kkuhi9m0sqio4f7e5l3q60idg7kouhdmeofo0dra3f8idfbekcval6k87/family1/f2770e92eb66401a94ef9b5748d28960 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/613b727b7ef448ab88c4a33e069ed0c8_SeqId_53_ 2023-02-12 14:55:06,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q0464m8kkuhi9m0sqio4f7e5l3q60idg7kouhdmeofo0dra3f8idfbekcval6k87/family1/f2770e92eb66401a94ef9b5748d28960 into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/613b727b7ef448ab88c4a33e069ed0c8_SeqId_53_ - updating store file list. 2023-02-12 14:55:06,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/613b727b7ef448ab88c4a33e069ed0c8_SeqId_53_ into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:06,888 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q0464m8kkuhi9m0sqio4f7e5l3q60idg7kouhdmeofo0dra3f8idfbekcval6k87/family1/f2770e92eb66401a94ef9b5748d28960 into 8aa6e64a63f09f579f8ce0eebfb86e36/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/613b727b7ef448ab88c4a33e069ed0c8_SeqId_53_) 2023-02-12 14:55:06,889 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily__q0464m8kkuhi9m0sqio4f7e5l3q60idg7kouhdmeofo0dra3f8idfbekcval6k87/family1/f2770e92eb66401a94ef9b5748d28960 2023-02-12 14:55:06,891 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:06,891 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:06,892 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511207f041d6b5cdc414f9e9b28b2e203418d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e201511207f041d6b5cdc414f9e9b28b2e203418d 2023-02-12 14:55:06,893 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112839e6a205491a4af98ea14c845915d751 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/ab712011512d4414510a003ba5521696/family1/d41d8cd98f00b204e9800998ecf8427e2015112839e6a205491a4af98ea14c845915d751 2023-02-12 14:55:06,893 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2 expired mob files are deleted 2023-02-12 14:55:06,893 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-12 14:55:06,894 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,894 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,894 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:06,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:06,894 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:06,895 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=64, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:06,896 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 881 msec 2023-02-12 14:55:06,948 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$12(2441): Client=jenkins//172.31.14.131 modify table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily from 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} to 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} 2023-02-12 14:55:06,949 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=65, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:07,357 DEBUG [PEWorker-4] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-12 14:55:07,358 DEBUG [PEWorker-4] util.FSTableDescriptors(519): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000002 2023-02-12 14:55:07,358 INFO [PEWorker-4] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/.tabledesc/.tableinfo.0000000003 2023-02-12 14:55:07,360 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily}] 2023-02-12 14:55:07,362 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE}] 2023-02-12 14:55:07,363 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE 2023-02-12 14:55:07,363 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:07,363 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213707363"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213707363"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213707363"}]},"ts":"1676213707363"} 2023-02-12 14:55:07,365 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=68, ppid=67, state=RUNNABLE; CloseRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:07,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8aa6e64a63f09f579f8ce0eebfb86e36, disabling compactions & flushes 2023-02-12 14:55:07,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:07,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:07,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. after waiting 0 ms 2023-02-12 14:55:07,516 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:07,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-12 14:55:07,524 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:07,524 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8aa6e64a63f09f579f8ce0eebfb86e36: Waiting for close lock at 1676213707516Running coprocessor pre-close hooks at 1676213707516Disabling compacts and flushes for region at 1676213707516Disabling writes for close at 1676213707516Writing region close event to WAL at 1676213707521 (+5 ms)Running coprocessor post-close hooks at 1676213707524 (+3 ms)Closed at 1676213707524 2023-02-12 14:55:07,525 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 8aa6e64a63f09f579f8ce0eebfb86e36 to self. 2023-02-12 14:55:07,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,526 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=CLOSED 2023-02-12 14:55:07,526 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213707526"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213707526"}]},"ts":"1676213707526"} 2023-02-12 14:55:07,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=68, resume processing ppid=67 2023-02-12 14:55:07,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, ppid=67, state=SUCCESS; CloseRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001 in 162 msec 2023-02-12 14:55:07,529 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=67, ppid=66, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=true 2023-02-12 14:55:07,679 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:07,679 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213707679"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213707679"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213707679"}]},"ts":"1676213707679"} 2023-02-12 14:55:07,681 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=69, ppid=67, state=RUNNABLE; OpenRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:07,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:07,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 8aa6e64a63f09f579f8ce0eebfb86e36, NAME => 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:55:07,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:07,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,836 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,837 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:07,837 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1 2023-02-12 14:55:07,838 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8aa6e64a63f09f579f8ce0eebfb86e36 columnFamilyName family1 2023-02-12 14:55:07,842 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/5958189e9f614520a838583947cd0186 2023-02-12 14:55:07,845 DEBUG [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/family1/613b727b7ef448ab88c4a33e069ed0c8_SeqId_53_ 2023-02-12 14:55:07,845 INFO [StoreOpener-8aa6e64a63f09f579f8ce0eebfb86e36-1] regionserver.HStore(310): Store=8aa6e64a63f09f579f8ce0eebfb86e36/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:07,846 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,847 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:55:07,849 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 8aa6e64a63f09f579f8ce0eebfb86e36; next sequenceid=56; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68827419, jitterRate=0.02560846507549286}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:55:07,849 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 8aa6e64a63f09f579f8ce0eebfb86e36: Running coprocessor pre-open hook at 1676213707836Writing region info on filesystem at 1676213707836Initializing all the Stores at 1676213707836Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'} at 1676213707836Cleaning up temporary data from old regions at 1676213707847 (+11 ms)Cleaning up detritus from prior splits at 1676213707847Running coprocessor post-open hooks at 1676213707849 (+2 ms)Region opened successfully at 1676213707849 2023-02-12 14:55:07,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36., pid=69, masterSystemTime=1676213707832 2023-02-12 14:55:07,851 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:07,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:55:07,851 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=67 updating hbase:meta row=8aa6e64a63f09f579f8ce0eebfb86e36, regionState=OPEN, openSeqNum=56, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:07,851 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36.","families":{"info":[{"qualifier":"regioninfo","vlen":95,"tag":[],"timestamp":"1676213707851"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213707851"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213707851"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213707851"}]},"ts":"1676213707851"} 2023-02-12 14:55:07,854 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=69, resume processing ppid=67 2023-02-12 14:55:07,854 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=69, ppid=67, state=SUCCESS; OpenRegionProcedure 8aa6e64a63f09f579f8ce0eebfb86e36, server=jenkins-hbase4.apache.org,45319,1676213617001 in 172 msec 2023-02-12 14:55:07,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=66 2023-02-12 14:55:07,855 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=66, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily, region=8aa6e64a63f09f579f8ce0eebfb86e36, REOPEN/MOVE in 492 msec 2023-02-12 14:55:07,856 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-12 14:55:07,857 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; ReopenTableRegionsProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 495 msec 2023-02-12 14:55:07,858 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=65, state=SUCCESS; ModifyTableProcedure table=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily in 909 msec 2023-02-12 14:55:07,858 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=65 2023-02-12 14:55:07,860 INFO [Listener at localhost/37945] compactions.TestMobCompactor(278): alter status finished 2023-02-12 14:55:07,860 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'DAILY'}] of the table testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily 2023-02-12 14:55:07,861 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:07,862 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=70, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:07,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:07,863 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=70, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214307863 2023-02-12 14:55:07,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:55:07,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 4 selected files, and 0 irrelevant files table 'testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column 'family1' 2023-02-12 14:55:07,863 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:07,864 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x03697e22 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:07,868 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8d32f2a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151128 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151205 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160103 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150907 for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' and column='family1' 2023-02-12 14:55:07,868 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily' column='family1' 2023-02-12 14:55:07,868 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:07,869 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=70, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE 2023-02-12 14:55:07,870 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=70, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily.mobLock, type=EXCLUSIVE in 8 msec 2023-02-12 14:55:07,925 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily Thread=252 (was 242) - Thread LEAK? -, OpenFileDescriptor=643 (was 610) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=181 (was 192), ProcessCount=168 (was 168), AvailableMemoryMB=3868 (was 3882) 2023-02-12 14:55:07,936 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=252, OpenFileDescriptor=643, MaxFileDescriptor=60000, SystemLoadAverage=181, ProcessCount=168, AvailableMemoryMB=3867 2023-02-12 14:55:07,938 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-12 14:55:07,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:07,940 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:55:07,941 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testGetAfterCompaction" procId is: 71 2023-02-12 14:55:07,941 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:55:07,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-12 14:55:07,942 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:07,943 DEBUG [HFileArchiver-11] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28 empty. 2023-02-12 14:55:07,943 DEBUG [HFileArchiver-11] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:07,943 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-12 14:55:08,042 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-12 14:55:08,243 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-12 14:55:08,350 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/.tabledesc/.tableinfo.0000000001 2023-02-12 14:55:08,351 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => f8d76535b6d60c5cd1522f78cc30cb28, NAME => 'testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testGetAfterCompaction', {TABLE_ATTRIBUTES => {coprocessor$1 => '|org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro|1073741823|'}}, {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:55:08,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-12 14:55:08,757 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:08,757 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1603): Closing f8d76535b6d60c5cd1522f78cc30cb28, disabling compactions & flushes 2023-02-12 14:55:08,757 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:08,757 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:08,757 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. after waiting 0 ms 2023-02-12 14:55:08,757 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:08,757 INFO [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:08,757 DEBUG [RegionOpenAndInit-testGetAfterCompaction-pool-0] regionserver.HRegion(1557): Region close journal for f8d76535b6d60c5cd1522f78cc30cb28: Waiting for close lock at 1676213708757Disabling compacts and flushes for region at 1676213708757Disabling writes for close at 1676213708757Writing region close event to WAL at 1676213708757Closed at 1676213708757 2023-02-12 14:55:08,759 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:55:08,760 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676213708760"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213708760"}]},"ts":"1676213708760"} 2023-02-12 14:55:08,761 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:55:08,762 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:55:08,762 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213708762"}]},"ts":"1676213708762"} 2023-02-12 14:55:08,763 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLING in hbase:meta 2023-02-12 14:55:08,766 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f8d76535b6d60c5cd1522f78cc30cb28, ASSIGN}] 2023-02-12 14:55:08,768 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f8d76535b6d60c5cd1522f78cc30cb28, ASSIGN 2023-02-12 14:55:08,768 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=72, ppid=71, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testGetAfterCompaction, region=f8d76535b6d60c5cd1522f78cc30cb28, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:55:08,919 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f8d76535b6d60c5cd1522f78cc30cb28, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:08,919 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676213708919"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213708919"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213708919"}]},"ts":"1676213708919"} 2023-02-12 14:55:08,921 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE; OpenRegionProcedure f8d76535b6d60c5cd1522f78cc30cb28, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:09,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-12 14:55:09,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:09,074 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => f8d76535b6d60c5cd1522f78cc30cb28, NAME => 'testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:55:09,075 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro with path null and priority 1073741823 2023-02-12 14:55:09,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro from HTD of testGetAfterCompaction successfully. 2023-02-12 14:55:09,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testGetAfterCompaction f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:09,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:09,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:09,076 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:09,077 INFO [StoreOpener-f8d76535b6d60c5cd1522f78cc30cb28-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family f1 of region f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:09,079 DEBUG [StoreOpener-f8d76535b6d60c5cd1522f78cc30cb28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1 2023-02-12 14:55:09,079 DEBUG [StoreOpener-f8d76535b6d60c5cd1522f78cc30cb28-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1 2023-02-12 14:55:09,079 INFO [StoreOpener-f8d76535b6d60c5cd1522f78cc30cb28-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f8d76535b6d60c5cd1522f78cc30cb28 columnFamilyName f1 2023-02-12 14:55:09,080 INFO [StoreOpener-f8d76535b6d60c5cd1522f78cc30cb28-1] regionserver.HStore(310): Store=f8d76535b6d60c5cd1522f78cc30cb28/f1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:09,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:09,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:09,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:09,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:55:09,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened f8d76535b6d60c5cd1522f78cc30cb28; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61811627, jitterRate=-0.07893498241901398}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:55:09,085 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for f8d76535b6d60c5cd1522f78cc30cb28: Running coprocessor pre-open hook at 1676213709076Writing region info on filesystem at 1676213709076Initializing all the Stores at 1676213709077 (+1 ms)Instantiating store for column family {NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213709077Cleaning up temporary data from old regions at 1676213709081 (+4 ms)Cleaning up detritus from prior splits at 1676213709082 (+1 ms)Running coprocessor post-open hooks at 1676213709085 (+3 ms)Region opened successfully at 1676213709085 2023-02-12 14:55:09,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28., pid=73, masterSystemTime=1676213709072 2023-02-12 14:55:09,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:09,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:09,087 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=72 updating hbase:meta row=f8d76535b6d60c5cd1522f78cc30cb28, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:09,088 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676213709087"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213709087"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213709087"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213709087"}]},"ts":"1676213709087"} 2023-02-12 14:55:09,090 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-12 14:55:09,090 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; OpenRegionProcedure f8d76535b6d60c5cd1522f78cc30cb28, server=jenkins-hbase4.apache.org,45319,1676213617001 in 168 msec 2023-02-12 14:55:09,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=72, resume processing ppid=71 2023-02-12 14:55:09,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, ppid=71, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f8d76535b6d60c5cd1522f78cc30cb28, ASSIGN in 324 msec 2023-02-12 14:55:09,092 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:55:09,092 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213709092"}]},"ts":"1676213709092"} 2023-02-12 14:55:09,093 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=ENABLED in hbase:meta 2023-02-12 14:55:09,095 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=71, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testGetAfterCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:55:09,096 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=71, state=SUCCESS; CreateTableProcedure table=testGetAfterCompaction in 1.1560 sec 2023-02-12 14:55:10,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=71 2023-02-12 14:55:10,046 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testGetAfterCompaction, procId: 71 completed 2023-02-12 14:55:10,046 DEBUG [Listener at localhost/37945] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testGetAfterCompaction get assigned. Timeout = 60000ms 2023-02-12 14:55:10,048 INFO [Listener at localhost/37945] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-12 14:55:10,051 INFO [Listener at localhost/37945] hbase.HBaseTestingUtility(3484): All regions for table testGetAfterCompaction assigned to meta. Checking AM states. 2023-02-12 14:55:10,051 INFO [Listener at localhost/37945] hbase.Waiter(180): Waiting up to [60,000] milli-secs(wait.for.ratio=[1]) 2023-02-12 14:55:10,051 INFO [Listener at localhost/37945] hbase.HBaseTestingUtility(3504): All regions for table testGetAfterCompaction assigned. 2023-02-12 14:55:10,055 INFO [Listener at localhost/37945] regionserver.HRegion(2744): Flushing f8d76535b6d60c5cd1522f78cc30cb28 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-12 14:55:10,466 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302120a54a6ee964c495e810eaf6927b0c252 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302120a54a6ee964c495e810eaf6927b0c252 2023-02-12 14:55:10,731 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:55:10,731 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x199904bc to 127.0.0.1:61565 2023-02-12 14:55:10,731 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:10,732 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x03697e22 to 127.0.0.1:61565 2023-02-12 14:55:10,732 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:10,732 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:55:10,732 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x047264c2 to 127.0.0.1:61565 2023-02-12 14:55:10,732 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:10,732 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:55:10,732 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x485ae223 to 127.0.0.1:61565 2023-02-12 14:55:10,732 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:10,732 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4cff90d4 to 127.0.0.1:61565 2023-02-12 14:55:10,732 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:10,732 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:55:10,733 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x4ba3f36e to 127.0.0.1:61565 2023-02-12 14:55:10,733 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:10,871 INFO [Listener at localhost/37945] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/.tmp/f1/19348b1fc87148ec90b7513ae9e6d1a4 2023-02-12 14:55:10,876 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/.tmp/f1/19348b1fc87148ec90b7513ae9e6d1a4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4 2023-02-12 14:55:10,879 INFO [Listener at localhost/37945] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4, entries=1, sequenceid=5, filesize=4.9 K 2023-02-12 14:55:10,880 INFO [Listener at localhost/37945] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f8d76535b6d60c5cd1522f78cc30cb28 in 825ms, sequenceid=5, compaction requested=false 2023-02-12 14:55:10,880 DEBUG [Listener at localhost/37945] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testGetAfterCompaction' 2023-02-12 14:55:10,881 DEBUG [Listener at localhost/37945] regionserver.HRegion(2445): Flush status journal for f8d76535b6d60c5cd1522f78cc30cb28: 2023-02-12 14:55:10,882 INFO [Listener at localhost/37945] regionserver.HRegion(2744): Flushing f8d76535b6d60c5cd1522f78cc30cb28 1/1 column families, dataSize=37 B heapSize=368 B 2023-02-12 14:55:11,239 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:55:11,294 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302127bdda059f774418f89867fc738e9b763 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302127bdda059f774418f89867fc738e9b763 2023-02-12 14:55:11,700 INFO [Listener at localhost/37945] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=37, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/.tmp/f1/7334dca9c5024c218168ea76f427c029 2023-02-12 14:55:11,704 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/.tmp/f1/7334dca9c5024c218168ea76f427c029 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029 2023-02-12 14:55:11,707 INFO [Listener at localhost/37945] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029, entries=1, sequenceid=9, filesize=4.9 K 2023-02-12 14:55:11,708 INFO [Listener at localhost/37945] regionserver.HRegion(2947): Finished flush of dataSize ~37 B/37, heapSize ~352 B/352, currentSize=0 B/0 for f8d76535b6d60c5cd1522f78cc30cb28 in 826ms, sequenceid=9, compaction requested=false 2023-02-12 14:55:11,708 DEBUG [Listener at localhost/37945] regionserver.HRegion(2445): Flush status journal for f8d76535b6d60c5cd1522f78cc30cb28: 2023-02-12 14:55:11,710 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'f1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testGetAfterCompaction 2023-02-12 14:55:11,711 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-12 14:55:11,711 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=74, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-12 14:55:11,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:11,712 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=74, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214311712 2023-02-12 14:55:11,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-12 14:55:11,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 2 selected files, and 0 irrelevant files table 'testGetAfterCompaction' and column 'f1' 2023-02-12 14:55:11,712 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testGetAfterCompaction' column='f1' 2023-02-12 14:55:11,713 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x53bf1552 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:11,716 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4284ca70, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:11,716 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230212 for table='testGetAfterCompaction' and column='f1' 2023-02-12 14:55:11,761 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,761 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,812 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,812 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,862 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,862 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,913 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,913 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,964 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:11,964 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,015 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,015 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,066 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,066 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,116 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,116 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,167 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,167 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,218 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,218 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,269 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,269 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,319 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,319 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,370 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,370 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,421 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,421 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,472 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,472 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,522 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,522 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,538 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212f5c15f0b463d4674ad4b4ad02286f262 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230212f5c15f0b463d4674ad4b4ad02286f262 2023-02-12 14:55:12,539 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testGetAfterCompaction' column='f1' 2023-02-12 14:55:12,551 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:12,552 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42180, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:12,558 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:55:12,560 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40524, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:55:12,567 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/f1/e558c2fd6ea5454f8571cadf00d500dc first=Optional[r1] last=Optional[r2] 2023-02-12 14:55:12,568 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{f1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/f1/e558c2fd6ea5454f8571cadf00d500dc}] 2023-02-12 14:55:12,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/f1/e558c2fd6ea5454f8571cadf00d500dc for inclusion in f8d76535b6d60c5cd1522f78cc30cb28/f1 2023-02-12 14:55:12,573 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,573 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r1 last=r2 2023-02-12 14:55:12,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:55:12,573 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for f8d76535b6d60c5cd1522f78cc30cb28: 2023-02-12 14:55:12,574 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/f1/e558c2fd6ea5454f8571cadf00d500dc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testGetAfterCompaction__5rsntqqslau0ob7u3l72f8151kiiuvvu2q6c51810l4mocvl1oluoh2pjt5dftq2/f1/e558c2fd6ea5454f8571cadf00d500dc 2023-02-12 14:55:12,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testGetAfterCompaction__5rsntqqslau0ob7u3l72f8151kiiuvvu2q6c51810l4mocvl1oluoh2pjt5dftq2/f1/e558c2fd6ea5454f8571cadf00d500dc as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_ 2023-02-12 14:55:12,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testGetAfterCompaction__5rsntqqslau0ob7u3l72f8151kiiuvvu2q6c51810l4mocvl1oluoh2pjt5dftq2/f1/e558c2fd6ea5454f8571cadf00d500dc into f8d76535b6d60c5cd1522f78cc30cb28/f1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_ - updating store file list. 2023-02-12 14:55:12,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_ into f8d76535b6d60c5cd1522f78cc30cb28/f1 2023-02-12 14:55:12,579 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testGetAfterCompaction__5rsntqqslau0ob7u3l72f8151kiiuvvu2q6c51810l4mocvl1oluoh2pjt5dftq2/f1/e558c2fd6ea5454f8571cadf00d500dc into f8d76535b6d60c5cd1522f78cc30cb28/f1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_) 2023-02-12 14:55:12,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testGetAfterCompaction__5rsntqqslau0ob7u3l72f8151kiiuvvu2q6c51810l4mocvl1oluoh2pjt5dftq2/f1/e558c2fd6ea5454f8571cadf00d500dc 2023-02-12 14:55:12,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): f8d76535b6d60c5cd1522f78cc30cb28/f1 is initiating minor compaction 2023-02-12 14:55:12,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@3fe5669b; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:12,580 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of f8d76535b6d60c5cd1522f78cc30cb28/f1 in testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:12,581 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/.tmp, totalSize=10.0 K 2023-02-12 14:55:12,581 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 7334dca9c5024c218168ea76f427c029, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:55:12,581 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:55:12,583 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testGetAfterCompaction' column='f1' 2023-02-12 14:55:12,584 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:12,585 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302120a54a6ee964c495e810eaf6927b0c252 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302120a54a6ee964c495e810eaf6927b0c252 2023-02-12 14:55:12,586 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302127bdda059f774418f89867fc738e9b763 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e202302127bdda059f774418f89867fc738e9b763 2023-02-12 14:55:12,586 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testGetAfterCompaction 2 expired mob files are deleted 2023-02-12 14:55:12,586 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testGetAfterCompaction/d41d8cd98f00b204e9800998ecf8427e20230212' 2023-02-12 14:55:12,587 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230212 for table='testGetAfterCompaction' and column='f1' 2023-02-12 14:55:12,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 1 mob files. table='testGetAfterCompaction' column='f1' 2023-02-12 14:55:12,587 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testGetAfterCompaction', column='f1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:55:12,588 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=74, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE 2023-02-12 14:55:12,589 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): f8d76535b6d60c5cd1522f78cc30cb28#f1#compaction#76 average throughput is 0.13 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:12,589 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=74, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testGetAfterCompaction.mobLock, type=EXCLUSIVE in 878 msec 2023-02-12 14:55:12,624 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,624 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,624 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,627 DEBUG [Listener at localhost/37945] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-12 14:55:12,630 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42194, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-12 14:55:12,631 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45319] regionserver.RSRpcServices(1662): Compacting testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:12,631 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 3 store files, 2 compacting, 0 eligible, 16 blocking 2023-02-12 14:55:12,631 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-12 14:55:12,631 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45319] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-12 14:55:12,631 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=45319] regionserver.CompactSplit(415): Not compacting testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. because compaction request was cancelled 2023-02-12 14:55:12,683 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,683 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,683 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,734 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,734 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,734 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,784 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,784 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,784 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,835 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,835 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,835 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,886 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,886 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,886 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,937 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,937 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,937 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,987 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:12,987 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:12,988 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,038 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,038 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,038 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,089 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,089 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,089 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,140 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,140 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,140 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,190 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,191 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,191 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,241 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,241 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,241 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,292 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,292 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,292 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,343 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,343 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,343 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,394 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,394 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,394 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,402 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/.tmp/f1/fd9c0d678e3a48adb767c46c571e5384 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/fd9c0d678e3a48adb767c46c571e5384 2023-02-12 14:55:13,406 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 2 file(s) in f8d76535b6d60c5cd1522f78cc30cb28/f1 of f8d76535b6d60c5cd1522f78cc30cb28 into fd9c0d678e3a48adb767c46c571e5384(size=5.1 K), total size for store is 10.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:13,406 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for f8d76535b6d60c5cd1522f78cc30cb28: 2023-02-12 14:55:13,406 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28., storeName=f1, fileCount=2, fileSize=10.0 K (4.9 K, 5.1 K), priority=2, time=1676213712580; duration=0sec 2023-02-12 14:55:13,406 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:13,444 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,445 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_; isDirectory=false; length=5206; replication=1; blocksize=134217728; modification_time=1676213712534; access_time=1676213711726; owner=jenkins; group=supergroup; permission=rwxrwxrwx; isSymlink=false} 2023-02-12 14:55:13,445 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213711699; access_time=1676213710884; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,445 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/fd9c0d678e3a48adb767c46c571e5384; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676213713397; access_time=1676213712586; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_] to archive 2023-02-12 14:55:13,450 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:13,452 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/7334dca9c5024c218168ea76f427c029 2023-02-12 14:55:13,453 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/6f1d4accd8c8488ea1e1fb1df3397973_SeqId_12_ 2023-02-12 14:55:13,495 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4; isDirectory=false; length=5015; replication=1; blocksize=134217728; modification_time=1676213710871; access_time=1676213710058; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,495 INFO [Listener at localhost/37945] compactions.TestMobCompactor(720): FileStatus{path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/fd9c0d678e3a48adb767c46c571e5384; isDirectory=false; length=5228; replication=1; blocksize=134217728; modification_time=1676213713397; access_time=1676213712586; owner=jenkins.hfs.0; group=supergroup; permission=rw-r--r--; isSymlink=false} 2023-02-12 14:55:13,612 INFO [Listener at localhost/37945] client.HBaseAdmin$15(890): Started disable of testGetAfterCompaction 2023-02-12 14:55:13,616 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testGetAfterCompaction 2023-02-12 14:55:13,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=75, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:13,624 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213713623"}]},"ts":"1676213713623"} 2023-02-12 14:55:13,625 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLING in hbase:meta 2023-02-12 14:55:13,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-12 14:55:13,629 INFO [PEWorker-3] procedure.DisableTableProcedure(293): Set testGetAfterCompaction to state=DISABLING 2023-02-12 14:55:13,630 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f8d76535b6d60c5cd1522f78cc30cb28, UNASSIGN}] 2023-02-12 14:55:13,631 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=75, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testGetAfterCompaction, region=f8d76535b6d60c5cd1522f78cc30cb28, UNASSIGN 2023-02-12 14:55:13,632 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f8d76535b6d60c5cd1522f78cc30cb28, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:13,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676213713632"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213713632"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213713632"}]},"ts":"1676213713632"} 2023-02-12 14:55:13,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; CloseRegionProcedure f8d76535b6d60c5cd1522f78cc30cb28, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:13,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-12 14:55:13,784 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:13,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing f8d76535b6d60c5cd1522f78cc30cb28, disabling compactions & flushes 2023-02-12 14:55:13,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:13,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:13,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. after waiting 0 ms 2023-02-12 14:55:13,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:13,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-12 14:55:13,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.mob.compactions.TestMobCompactor$CompactTwoLatestHfilesCopro 2023-02-12 14:55:13,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28. 2023-02-12 14:55:13,790 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for f8d76535b6d60c5cd1522f78cc30cb28: Waiting for close lock at 1676213713785Running coprocessor pre-close hooks at 1676213713785Disabling compacts and flushes for region at 1676213713785Disabling writes for close at 1676213713785Writing region close event to WAL at 1676213713785Running coprocessor post-close hooks at 1676213713789 (+4 ms)Closed at 1676213713790 (+1 ms) 2023-02-12 14:55:13,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:13,791 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=f8d76535b6d60c5cd1522f78cc30cb28, regionState=CLOSED 2023-02-12 14:55:13,791 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676213713791"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213713791"}]},"ts":"1676213713791"} 2023-02-12 14:55:13,793 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-02-12 14:55:13,794 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; CloseRegionProcedure f8d76535b6d60c5cd1522f78cc30cb28, server=jenkins-hbase4.apache.org,45319,1676213617001 in 159 msec 2023-02-12 14:55:13,794 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=75 2023-02-12 14:55:13,795 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=75, state=SUCCESS; TransitRegionStateProcedure table=testGetAfterCompaction, region=f8d76535b6d60c5cd1522f78cc30cb28, UNASSIGN in 164 msec 2023-02-12 14:55:13,795 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213713795"}]},"ts":"1676213713795"} 2023-02-12 14:55:13,796 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testGetAfterCompaction, state=DISABLED in hbase:meta 2023-02-12 14:55:13,797 INFO [PEWorker-3] procedure.DisableTableProcedure(305): Set testGetAfterCompaction to state=DISABLED 2023-02-12 14:55:13,798 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=75, state=SUCCESS; DisableTableProcedure table=testGetAfterCompaction in 181 msec 2023-02-12 14:55:13,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=75 2023-02-12 14:55:13,927 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testGetAfterCompaction, procId: 75 completed 2023-02-12 14:55:13,931 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$5(2215): Client=jenkins//172.31.14.131 delete testGetAfterCompaction 2023-02-12 14:55:13,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:13,938 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=78, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:13,939 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=78, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:13,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-12 14:55:13,942 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:13,944 DEBUG [HFileArchiver-12] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1, FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/recovered.edits] 2023-02-12 14:55:13,948 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/19348b1fc87148ec90b7513ae9e6d1a4 2023-02-12 14:55:13,949 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/fd9c0d678e3a48adb767c46c571e5384 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/f1/fd9c0d678e3a48adb767c46c571e5384 2023-02-12 14:55:13,952 DEBUG [HFileArchiver-12] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/recovered.edits/15.seqid to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28/recovered.edits/15.seqid 2023-02-12 14:55:13,952 DEBUG [HFileArchiver-12] backup.HFileArchiver(596): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testGetAfterCompaction/f8d76535b6d60c5cd1522f78cc30cb28 2023-02-12 14:55:13,952 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(328): Archived testGetAfterCompaction regions 2023-02-12 14:55:13,953 DEBUG [PEWorker-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-12 14:55:13,953 DEBUG [PEWorker-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1] 2023-02-12 14:55:13,956 DEBUG [PEWorker-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230212f5c15f0b463d4674ad4b4ad02286f262 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0/f1/d41d8cd98f00b204e9800998ecf8427e20230212f5c15f0b463d4674ad4b4ad02286f262 2023-02-12 14:55:13,956 DEBUG [PEWorker-4] backup.HFileArchiver(596): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testGetAfterCompaction/324ed642d7db99a22ebbb8cbc71c05d0 2023-02-12 14:55:13,958 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=78, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:13,979 WARN [PEWorker-4] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testGetAfterCompaction from hbase:meta 2023-02-12 14:55:13,981 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(421): Removing 'testGetAfterCompaction' descriptor. 2023-02-12 14:55:13,982 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=78, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:13,982 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(411): Removing 'testGetAfterCompaction' from region states. 2023-02-12 14:55:13,982 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676213713982"}]},"ts":"9223372036854775807"} 2023-02-12 14:55:13,984 INFO [PEWorker-4] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-12 14:55:13,984 DEBUG [PEWorker-4] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f8d76535b6d60c5cd1522f78cc30cb28, NAME => 'testGetAfterCompaction,,1676213707938.f8d76535b6d60c5cd1522f78cc30cb28.', STARTKEY => '', ENDKEY => ''}] 2023-02-12 14:55:13,984 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(415): Marking 'testGetAfterCompaction' as deleted. 2023-02-12 14:55:13,984 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testGetAfterCompaction","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676213713984"}]},"ts":"9223372036854775807"} 2023-02-12 14:55:13,988 INFO [PEWorker-4] hbase.MetaTableAccessor(1658): Deleted table testGetAfterCompaction state from META 2023-02-12 14:55:13,992 DEBUG [PEWorker-4] procedure.DeleteTableProcedure(130): Finished pid=78, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testGetAfterCompaction 2023-02-12 14:55:13,993 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=78, state=SUCCESS; DeleteTableProcedure table=testGetAfterCompaction in 60 msec 2023-02-12 14:55:14,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=78 2023-02-12 14:55:14,041 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testGetAfterCompaction, procId: 78 completed 2023-02-12 14:55:14,054 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testGetAfterCompaction Thread=236 (was 252), OpenFileDescriptor=600 (was 643), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=166 (was 181), ProcessCount=168 (was 168), AvailableMemoryMB=3835 (was 3867) 2023-02-12 14:55:14,065 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=236, OpenFileDescriptor=600, MaxFileDescriptor=60000, SystemLoadAverage=166, ProcessCount=168, AvailableMemoryMB=3835 2023-02-12 14:55:14,067 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-12 14:55:14,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testCompactionWithHFileLink 2023-02-12 14:55:14,068 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:55:14,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testCompactionWithHFileLink" procId is: 79 2023-02-12 14:55:14,069 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:55:14,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 14:55:14,071 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:14,071 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:14,071 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:14,072 DEBUG [HFileArchiver-13] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8 empty. 2023-02-12 14:55:14,072 DEBUG [HFileArchiver-15] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea empty. 2023-02-12 14:55:14,072 DEBUG [HFileArchiver-14] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6 empty. 2023-02-12 14:55:14,072 DEBUG [HFileArchiver-13] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:14,072 DEBUG [HFileArchiver-14] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:14,072 DEBUG [HFileArchiver-15] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:14,073 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testCompactionWithHFileLink regions 2023-02-12 14:55:14,170 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 14:55:14,371 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 14:55:14,480 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-12 14:55:14,481 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(7671): creating {ENCODED => 73ab03d290bf433ec02678876bd209ea, NAME => 'testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:55:14,482 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(7671): creating {ENCODED => c330f294c6804231199503c93a0f53d8, NAME => 'testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:55:14,482 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(7671): creating {ENCODED => 7c8a55283e262bc196e8394e111484b6, NAME => 'testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testCompactionWithHFileLink', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:55:14,672 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 14:55:14,890 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:14,890 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:14,890 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1603): Closing c330f294c6804231199503c93a0f53d8, disabling compactions & flushes 2023-02-12 14:55:14,890 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1603): Closing 73ab03d290bf433ec02678876bd209ea, disabling compactions & flushes 2023-02-12 14:55:14,890 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:14,890 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:14,890 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:14,890 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1603): Closing 7c8a55283e262bc196e8394e111484b6, disabling compactions & flushes 2023-02-12 14:55:14,890 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. after waiting 0 ms 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:14,891 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. after waiting 0 ms 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:14,891 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:14,891 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-0] regionserver.HRegion(1557): Region close journal for c330f294c6804231199503c93a0f53d8: Waiting for close lock at 1676213714890Disabling compacts and flushes for region at 1676213714890Disabling writes for close at 1676213714891 (+1 ms)Writing region close event to WAL at 1676213714891Closed at 1676213714891 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-1] regionserver.HRegion(1557): Region close journal for 7c8a55283e262bc196e8394e111484b6: Waiting for close lock at 1676213714891Disabling compacts and flushes for region at 1676213714891Disabling writes for close at 1676213714891Writing region close event to WAL at 1676213714891Closed at 1676213714891 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. after waiting 0 ms 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:14,891 INFO [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:14,891 DEBUG [RegionOpenAndInit-testCompactionWithHFileLink-pool-2] regionserver.HRegion(1557): Region close journal for 73ab03d290bf433ec02678876bd209ea: Waiting for close lock at 1676213714890Disabling compacts and flushes for region at 1676213714890Disabling writes for close at 1676213714891 (+1 ms)Writing region close event to WAL at 1676213714891Closed at 1676213714891 2023-02-12 14:55:14,893 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:55:14,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213714894"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213714894"}]},"ts":"1676213714894"} 2023-02-12 14:55:14,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213714894"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213714894"}]},"ts":"1676213714894"} 2023-02-12 14:55:14,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213714894"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213714894"}]},"ts":"1676213714894"} 2023-02-12 14:55:14,896 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-12 14:55:14,896 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:55:14,897 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213714896"}]},"ts":"1676213714896"} 2023-02-12 14:55:14,898 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-12 14:55:14,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN}, {pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN}] 2023-02-12 14:55:14,903 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN 2023-02-12 14:55:14,903 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN 2023-02-12 14:55:14,903 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN 2023-02-12 14:55:14,903 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=82, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:55:14,903 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:55:14,904 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:55:15,055 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=c330f294c6804231199503c93a0f53d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:15,055 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=73ab03d290bf433ec02678876bd209ea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:15,055 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=7c8a55283e262bc196e8394e111484b6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:15,055 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213715055"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213715055"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213715055"}]},"ts":"1676213715055"} 2023-02-12 14:55:15,055 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213715055"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213715055"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213715055"}]},"ts":"1676213715055"} 2023-02-12 14:55:15,055 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213715055"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213715055"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213715055"}]},"ts":"1676213715055"} 2023-02-12 14:55:15,057 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; OpenRegionProcedure c330f294c6804231199503c93a0f53d8, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:15,057 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=84, ppid=82, state=RUNNABLE; OpenRegionProcedure 73ab03d290bf433ec02678876bd209ea, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:15,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=85, ppid=81, state=RUNNABLE; OpenRegionProcedure 7c8a55283e262bc196e8394e111484b6, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:15,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 14:55:15,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:15,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c330f294c6804231199503c93a0f53d8, NAME => 'testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.', STARTKEY => '', ENDKEY => '1'} 2023-02-12 14:55:15,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:15,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,214 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,215 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:15,215 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:15,215 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c330f294c6804231199503c93a0f53d8 columnFamilyName family1 2023-02-12 14:55:15,216 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(310): Store=c330f294c6804231199503c93a0f53d8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:15,217 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,218 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2 2023-02-12 14:55:15,218 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2 2023-02-12 14:55:15,218 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c330f294c6804231199503c93a0f53d8 columnFamilyName family2 2023-02-12 14:55:15,219 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(310): Store=c330f294c6804231199503c93a0f53d8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:15,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,220 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,221 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:15,222 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:15,223 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:55:15,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c330f294c6804231199503c93a0f53d8; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59835755, jitterRate=-0.10837776958942413}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:15,224 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c330f294c6804231199503c93a0f53d8: Running coprocessor pre-open hook at 1676213715213Writing region info on filesystem at 1676213715213Initializing all the Stores at 1676213715214 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213715214Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213715214Cleaning up temporary data from old regions at 1676213715220 (+6 ms)Cleaning up detritus from prior splits at 1676213715221 (+1 ms)Running coprocessor post-open hooks at 1676213715224 (+3 ms)Region opened successfully at 1676213715224 2023-02-12 14:55:15,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8., pid=83, masterSystemTime=1676213715210 2023-02-12 14:55:15,225 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:15,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:15,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:15,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 73ab03d290bf433ec02678876bd209ea, NAME => 'testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.', STARTKEY => '2', ENDKEY => ''} 2023-02-12 14:55:15,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,226 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=c330f294c6804231199503c93a0f53d8, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:15,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:15,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,226 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,226 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213715226"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213715226"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213715226"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213715226"}]},"ts":"1676213715226"} 2023-02-12 14:55:15,227 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,228 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:15,228 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:15,228 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73ab03d290bf433ec02678876bd209ea columnFamilyName family1 2023-02-12 14:55:15,229 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-12 14:55:15,229 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; OpenRegionProcedure c330f294c6804231199503c93a0f53d8, server=jenkins-hbase4.apache.org,45319,1676213617001 in 170 msec 2023-02-12 14:55:15,229 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(310): Store=73ab03d290bf433ec02678876bd209ea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:15,229 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN in 329 msec 2023-02-12 14:55:15,230 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2 2023-02-12 14:55:15,231 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2 2023-02-12 14:55:15,231 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73ab03d290bf433ec02678876bd209ea columnFamilyName family2 2023-02-12 14:55:15,232 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(310): Store=73ab03d290bf433ec02678876bd209ea/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:15,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:15,234 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:15,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:55:15,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 73ab03d290bf433ec02678876bd209ea; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59478545, jitterRate=-0.11370061337947845}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:15,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 73ab03d290bf433ec02678876bd209ea: Running coprocessor pre-open hook at 1676213715226Writing region info on filesystem at 1676213715226Initializing all the Stores at 1676213715227 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213715227Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213715227Cleaning up temporary data from old regions at 1676213715233 (+6 ms)Cleaning up detritus from prior splits at 1676213715233Running coprocessor post-open hooks at 1676213715236 (+3 ms)Region opened successfully at 1676213715236 2023-02-12 14:55:15,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea., pid=84, masterSystemTime=1676213715210 2023-02-12 14:55:15,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:15,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:15,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:15,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7c8a55283e262bc196e8394e111484b6, NAME => 'testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.', STARTKEY => '1', ENDKEY => '2'} 2023-02-12 14:55:15,238 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=82 updating hbase:meta row=73ab03d290bf433ec02678876bd209ea, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:15,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:15,238 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213715238"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213715238"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213715238"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213715238"}]},"ts":"1676213715238"} 2023-02-12 14:55:15,238 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,239 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,239 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,240 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:15,241 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:15,241 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=84, resume processing ppid=82 2023-02-12 14:55:15,241 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=84, ppid=82, state=SUCCESS; OpenRegionProcedure 73ab03d290bf433ec02678876bd209ea, server=jenkins-hbase4.apache.org,45319,1676213617001 in 182 msec 2023-02-12 14:55:15,241 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c8a55283e262bc196e8394e111484b6 columnFamilyName family1 2023-02-12 14:55:15,242 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(310): Store=7c8a55283e262bc196e8394e111484b6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:15,242 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN in 340 msec 2023-02-12 14:55:15,242 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,243 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2 2023-02-12 14:55:15,243 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2 2023-02-12 14:55:15,243 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c8a55283e262bc196e8394e111484b6 columnFamilyName family2 2023-02-12 14:55:15,244 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(310): Store=7c8a55283e262bc196e8394e111484b6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:15,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:15,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:15,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:55:15,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7c8a55283e262bc196e8394e111484b6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67341288, jitterRate=0.0034633874893188477}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:15,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7c8a55283e262bc196e8394e111484b6: Running coprocessor pre-open hook at 1676213715239Writing region info on filesystem at 1676213715239Initializing all the Stores at 1676213715239Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213715239Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213715239Cleaning up temporary data from old regions at 1676213715245 (+6 ms)Cleaning up detritus from prior splits at 1676213715246 (+1 ms)Running coprocessor post-open hooks at 1676213715249 (+3 ms)Region opened successfully at 1676213715249 2023-02-12 14:55:15,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6., pid=85, masterSystemTime=1676213715210 2023-02-12 14:55:15,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:15,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:15,251 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=7c8a55283e262bc196e8394e111484b6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:15,252 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213715251"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213715251"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213715251"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213715251"}]},"ts":"1676213715251"} 2023-02-12 14:55:15,254 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=85, resume processing ppid=81 2023-02-12 14:55:15,254 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=85, ppid=81, state=SUCCESS; OpenRegionProcedure 7c8a55283e262bc196e8394e111484b6, server=jenkins-hbase4.apache.org,45319,1676213617001 in 195 msec 2023-02-12 14:55:15,255 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-02-12 14:55:15,256 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN in 354 msec 2023-02-12 14:55:15,256 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:55:15,256 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213715256"}]},"ts":"1676213715256"} 2023-02-12 14:55:15,257 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-12 14:55:15,259 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=79, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testCompactionWithHFileLink execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:55:15,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; CreateTableProcedure table=testCompactionWithHFileLink in 1.1920 sec 2023-02-12 14:55:16,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-12 14:55:16,174 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testCompactionWithHFileLink, procId: 79 completed 2023-02-12 14:55:16,181 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:16,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:16,183 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:16,186 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:16,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-12 14:55:16,188 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-12 14:55:16,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:16,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-12 14:55:16,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-12 14:55:16,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:16,188 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:16,190 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:16,190 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:16,190 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:16,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:16,190 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:16,190 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:16,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:16,191 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:16,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:16,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:16,191 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-12 14:55:16,191 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-12 14:55:16,191 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-12 14:55:16,191 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:16,191 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-12 14:55:16,192 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:16,192 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:16,192 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:16,192 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. started... 2023-02-12 14:55:16,192 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:16,192 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. started... 2023-02-12 14:55:16,192 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:16,193 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c8a55283e262bc196e8394e111484b6 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:55:16,193 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 73ab03d290bf433ec02678876bd209ea 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:55:16,193 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. started... 2023-02-12 14:55:16,193 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c330f294c6804231199503c93a0f53d8 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:55:16,217 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 2023-02-12 14:55:16,217 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 2023-02-12 14:55:16,222 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/e03728432e5947eda80697c61f891d10 2023-02-12 14:55:16,233 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212af91ff60a0254fbfa2c8d9efcf4a313d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212af91ff60a0254fbfa2c8d9efcf4a313d 2023-02-12 14:55:16,615 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 2023-02-12 14:55:16,621 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/ecdc577f07da4cb1ad7ba1378a36e7d2 2023-02-12 14:55:16,638 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/f2a416bc30b44a53874f33431ceee564 2023-02-12 14:55:16,641 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/e03728432e5947eda80697c61f891d10 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10 2023-02-12 14:55:16,644 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10, entries=4, sequenceid=5, filesize=5.4 K 2023-02-12 14:55:16,645 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/f2a416bc30b44a53874f33431ceee564 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564 2023-02-12 14:55:16,648 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:55:16,648 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 73ab03d290bf433ec02678876bd209ea in 455ms, sequenceid=5, compaction requested=false 2023-02-12 14:55:16,648 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-12 14:55:16,649 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:16,649 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:16,649 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:17,020 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/69f858874fa141889a3f3d14d75bc3fd 2023-02-12 14:55:17,033 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212cc09253e7c7f4e1683b5d6f6775972fb to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cc09253e7c7f4e1683b5d6f6775972fb 2023-02-12 14:55:17,434 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302122cc3e41fa25843a1848eaeae94611883 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302122cc3e41fa25843a1848eaeae94611883 2023-02-12 14:55:17,437 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2 2023-02-12 14:55:17,441 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/ecdc577f07da4cb1ad7ba1378a36e7d2 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2 2023-02-12 14:55:17,445 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2, entries=4, sequenceid=5, filesize=5.4 K 2023-02-12 14:55:17,445 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2 2023-02-12 14:55:17,448 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:55:17,449 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 7c8a55283e262bc196e8394e111484b6 in 1256ms, sequenceid=5, compaction requested=false 2023-02-12 14:55:17,449 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:17,449 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:17,449 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:17,838 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/18484b7a53624894893277beb20e6d58 2023-02-12 14:55:17,842 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/69f858874fa141889a3f3d14d75bc3fd as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd 2023-02-12 14:55:17,846 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd, entries=4, sequenceid=5, filesize=5.4 K 2023-02-12 14:55:17,847 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/18484b7a53624894893277beb20e6d58 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58 2023-02-12 14:55:17,850 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:55:17,851 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c330f294c6804231199503c93a0f53d8 in 1658ms, sequenceid=5, compaction requested=false 2023-02-12 14:55:17,851 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:17,851 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:17,851 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:17,851 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:17,851 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,851 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-12 14:55:17,851 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-12 14:55:17,853 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,853 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:17,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:17,853 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,853 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:17,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:17,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:17,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:17,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:17,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-12 14:55:17,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f1a397a[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:17,854 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-12 14:55:17,854 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,855 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,855 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-12 14:55:17,855 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-12 14:55:17,855 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:17,855 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,855 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:17,856 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-12 14:55:17,856 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,857 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:17,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:17,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:17,857 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-12 14:55:17,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:17,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:17,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:17,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:17,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:17,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:17,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-12 14:55:17,858 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-12 14:55:17,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:17,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:17,859 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-12 14:55:17,859 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:17,861 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,861 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:17,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,861 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:17,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:17,862 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,863 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-12 14:55:17,865 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:17,865 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:17,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:17,865 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:17,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:17,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:17,965 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:17,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:17,967 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:17,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:17,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:17,970 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:17,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:17,972 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-12 14:55:17,972 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-12 14:55:17,972 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:17,972 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-12 14:55:17,972 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-12 14:55:17,972 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,972 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:17,973 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:17,973 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,973 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:17,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:17,974 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:17,974 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:17,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:17,974 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:17,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:17,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:17,974 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-12 14:55:17,974 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-12 14:55:17,975 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-12 14:55:17,975 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-12 14:55:17,975 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:17,975 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-12 14:55:17,975 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:17,975 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. started... 2023-02-12 14:55:17,975 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c8a55283e262bc196e8394e111484b6 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:55:17,977 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:17,977 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:17,977 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:17,977 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. started... 2023-02-12 14:55:17,977 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:17,977 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. started... 2023-02-12 14:55:17,977 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 73ab03d290bf433ec02678876bd209ea 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:55:17,977 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c330f294c6804231199503c93a0f53d8 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:55:18,395 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 2023-02-12 14:55:18,395 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 2023-02-12 14:55:18,398 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 2023-02-12 14:55:18,400 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/41be1e3dff234cadbea7808c2c672e0c 2023-02-12 14:55:18,804 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/5c12eba08d1240b9b5042b98c3072cf6 2023-02-12 14:55:18,808 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/49707ccb4e4f44f28a5e46acdf4a01f4 2023-02-12 14:55:18,817 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302123f15fb6bd0f04990aba6745eee75a1c9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123f15fb6bd0f04990aba6745eee75a1c9 2023-02-12 14:55:19,217 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302129b36e5cfbffa44e499012c03047dd37b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302129b36e5cfbffa44e499012c03047dd37b 2023-02-12 14:55:19,221 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021277eedf1d464140ec88991fce862497b9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021277eedf1d464140ec88991fce862497b9 2023-02-12 14:55:19,221 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/700fda40ac8649ef9a44f4cfdc556a9f 2023-02-12 14:55:19,222 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/a235bba083664fa998b43af265d0eed2 2023-02-12 14:55:19,227 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/41be1e3dff234cadbea7808c2c672e0c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c 2023-02-12 14:55:19,228 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/5c12eba08d1240b9b5042b98c3072cf6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6 2023-02-12 14:55:19,231 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c, entries=4, sequenceid=9, filesize=5.4 K 2023-02-12 14:55:19,231 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/700fda40ac8649ef9a44f4cfdc556a9f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f 2023-02-12 14:55:19,232 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6, entries=4, sequenceid=9, filesize=5.4 K 2023-02-12 14:55:19,233 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/a235bba083664fa998b43af265d0eed2 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2 2023-02-12 14:55:19,235 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:55:19,236 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c330f294c6804231199503c93a0f53d8 in 1259ms, sequenceid=9, compaction requested=false 2023-02-12 14:55:19,236 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:19,236 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:19,236 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:19,236 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:55:19,237 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 7c8a55283e262bc196e8394e111484b6 in 1262ms, sequenceid=9, compaction requested=false 2023-02-12 14:55:19,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:19,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:19,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:19,291 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:55:19,628 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/20f9cf39b0594acf9ec77c570d62698c 2023-02-12 14:55:19,632 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/49707ccb4e4f44f28a5e46acdf4a01f4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4 2023-02-12 14:55:19,635 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4, entries=4, sequenceid=9, filesize=5.4 K 2023-02-12 14:55:19,636 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/20f9cf39b0594acf9ec77c570d62698c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c 2023-02-12 14:55:19,639 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:55:19,639 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 73ab03d290bf433ec02678876bd209ea in 1662ms, sequenceid=9, compaction requested=false 2023-02-12 14:55:19,639 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:19,639 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-12 14:55:19,640 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-12 14:55:19,641 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,641 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:19,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:19,641 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,642 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:19,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:19,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:19,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:19,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:19,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-12 14:55:19,643 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f22841[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:19,643 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-12 14:55:19,643 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,645 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,645 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-12 14:55:19,645 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-12 14:55:19,645 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:19,645 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,645 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:19,647 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-12 14:55:19,647 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,647 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,647 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-12 14:55:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:19,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:19,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:19,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:19,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:19,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-12 14:55:19,649 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-12 14:55:19,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:19,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:19,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-12 14:55:19,649 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:19,650 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,650 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:19,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,650 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:19,651 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,651 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:19,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:19,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:19,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:19,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:19,652 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:19,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:19,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:19,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-12 14:55:19,654 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-12 14:55:19,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:19,654 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:19,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:19,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:19,654 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:19,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:19,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:19,754 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:19,755 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:19,756 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:19,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:19,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:19,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:19,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:19,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-12 14:55:19,761 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-12 14:55:19,761 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:19,761 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-12 14:55:19,761 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-12 14:55:19,761 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,761 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:19,763 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,763 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:19,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:19,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:19,764 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:19,764 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:19,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:19,764 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:19,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:19,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:19,764 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-12 14:55:19,765 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-12 14:55:19,765 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-12 14:55:19,765 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-12 14:55:19,765 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:19,765 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-12 14:55:19,765 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:19,765 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. started... 2023-02-12 14:55:19,765 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:19,765 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:19,765 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. started... 2023-02-12 14:55:19,765 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:19,765 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c330f294c6804231199503c93a0f53d8 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:55:19,765 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c8a55283e262bc196e8394e111484b6 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:55:19,765 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:19,765 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. started... 2023-02-12 14:55:19,765 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 73ab03d290bf433ec02678876bd209ea 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:55:19,789 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b 2023-02-12 14:55:20,186 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc 2023-02-12 14:55:20,186 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 2023-02-12 14:55:20,194 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/5ba1a1c8bb5046099d4c27b359ce7176 2023-02-12 14:55:20,591 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/526d1d45234641e5877c28f0ee1c4fe6 2023-02-12 14:55:20,591 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/a0644edaa9354d88938f406a3dbc2943 2023-02-12 14:55:20,605 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212d4be49120391415bb2ea6fe55808e6b9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212d4be49120391415bb2ea6fe55808e6b9 2023-02-12 14:55:21,005 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212283dbf899f7f49f1a537097648f293ad to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212283dbf899f7f49f1a537097648f293ad 2023-02-12 14:55:21,005 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302123748b931d9074f96b8c1f4dd73ecc85b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123748b931d9074f96b8c1f4dd73ecc85b 2023-02-12 14:55:21,009 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/30098feffd2e47808617c86382a508a5 2023-02-12 14:55:21,012 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/5ba1a1c8bb5046099d4c27b359ce7176 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176 2023-02-12 14:55:21,016 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176, entries=4, sequenceid=13, filesize=5.4 K 2023-02-12 14:55:21,017 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/30098feffd2e47808617c86382a508a5 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5 2023-02-12 14:55:21,019 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:55:21,020 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 7c8a55283e262bc196e8394e111484b6 in 1255ms, sequenceid=13, compaction requested=true 2023-02-12 14:55:21,020 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:21,020 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:21,020 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:21,410 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/f31845a05242422fb2c55797c8de37b4 2023-02-12 14:55:21,410 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/4aba417dd6d142129901d792ec2f7631 2023-02-12 14:55:21,415 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/a0644edaa9354d88938f406a3dbc2943 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943 2023-02-12 14:55:21,415 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/526d1d45234641e5877c28f0ee1c4fe6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6 2023-02-12 14:55:21,418 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943, entries=4, sequenceid=13, filesize=5.4 K 2023-02-12 14:55:21,418 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6, entries=4, sequenceid=13, filesize=5.4 K 2023-02-12 14:55:21,419 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/f31845a05242422fb2c55797c8de37b4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4 2023-02-12 14:55:21,419 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/4aba417dd6d142129901d792ec2f7631 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631 2023-02-12 14:55:21,422 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:55:21,422 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:55:21,423 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 73ab03d290bf433ec02678876bd209ea in 1658ms, sequenceid=13, compaction requested=true 2023-02-12 14:55:21,423 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c330f294c6804231199503c93a0f53d8 in 1658ms, sequenceid=13, compaction requested=true 2023-02-12 14:55:21,423 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:21,423 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:21,423 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:21,423 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:21,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:21,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:21,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:21,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-12 14:55:21,424 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-12 14:55:21,425 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:21,425 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:21,425 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,425 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:21,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-12 14:55:21,426 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5f1c5d8d[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:21,427 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-12 14:55:21,427 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,429 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,429 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-12 14:55:21,429 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-12 14:55:21,429 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,429 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:21,429 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:21,433 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-12 14:55:21,433 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:21,433 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,433 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-12 14:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:21,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:21,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:21,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:21,435 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-12 14:55:21,435 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-12 14:55:21,435 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:21,435 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:21,435 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-12 14:55:21,435 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:21,436 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,436 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,437 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:21,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:21,437 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:21,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:21,438 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:21,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:21,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:21,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:21,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:21,441 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:21,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-12 14:55:21,441 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:21,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:21,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:21,441 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:21,541 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:21,542 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:21,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:21,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:21,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:21,546 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:21,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:21,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-12 14:55:21,547 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-12 14:55:21,548 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:21,548 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-12 14:55:21,548 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-12 14:55:21,548 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,548 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:21,549 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,549 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:21,549 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:21,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:21,549 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:21,549 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:21,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:21,550 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:21,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:21,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:21,550 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-12 14:55:21,550 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testCompactionWithHFileLink is still present, removing 2023-02-12 14:55:21,550 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-12 14:55:21,550 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-12 14:55:21,550 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:21,550 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-12 14:55:21,558 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:21,558 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:21,558 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:21,558 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:21,558 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. started... 2023-02-12 14:55:21,558 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:21,558 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. started... 2023-02-12 14:55:21,558 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. started... 2023-02-12 14:55:21,558 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c330f294c6804231199503c93a0f53d8 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:55:21,559 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 73ab03d290bf433ec02678876bd209ea 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:55:21,559 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c8a55283e262bc196e8394e111484b6 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:55:21,984 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 2023-02-12 14:55:21,984 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff 2023-02-12 14:55:21,985 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf 2023-02-12 14:55:22,397 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/3e7296de83954248a5d807c9f40e18fa 2023-02-12 14:55:22,398 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/6f3d54677c014906aa0b5cb62516a2c3 2023-02-12 14:55:22,399 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/89aadb319ba44c88a9214d1ad6836b68 2023-02-12 14:55:22,819 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302125e1fe8592e164715b749d5d893383f7f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302125e1fe8592e164715b749d5d893383f7f 2023-02-12 14:55:22,819 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302120fb46ed2fc32453fbb3b6d6cec5237f6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302120fb46ed2fc32453fbb3b6d6cec5237f6 2023-02-12 14:55:22,819 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302123415c46c45d6470d9112000f8bf04c70 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302123415c46c45d6470d9112000f8bf04c70 2023-02-12 14:55:22,825 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/27b7d09543864486b8af9b45fb33b2f3 2023-02-12 14:55:22,828 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/89aadb319ba44c88a9214d1ad6836b68 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68 2023-02-12 14:55:22,831 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68, entries=4, sequenceid=17, filesize=5.4 K 2023-02-12 14:55:22,831 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/27b7d09543864486b8af9b45fb33b2f3 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3 2023-02-12 14:55:22,834 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:55:22,835 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 73ab03d290bf433ec02678876bd209ea in 1277ms, sequenceid=17, compaction requested=true 2023-02-12 14:55:22,835 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:22,835 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:22,835 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:23,224 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/66e244f61f03457384884137acf1b58b 2023-02-12 14:55:23,225 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/053d84dda82248ecaeb1d4d178f7710e 2023-02-12 14:55:23,229 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/3e7296de83954248a5d807c9f40e18fa as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa 2023-02-12 14:55:23,229 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/6f3d54677c014906aa0b5cb62516a2c3 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3 2023-02-12 14:55:23,232 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa, entries=4, sequenceid=17, filesize=5.4 K 2023-02-12 14:55:23,232 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3, entries=4, sequenceid=17, filesize=5.4 K 2023-02-12 14:55:23,233 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/66e244f61f03457384884137acf1b58b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b 2023-02-12 14:55:23,233 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/053d84dda82248ecaeb1d4d178f7710e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e 2023-02-12 14:55:23,236 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:55:23,236 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:55:23,237 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c330f294c6804231199503c93a0f53d8 in 1679ms, sequenceid=17, compaction requested=true 2023-02-12 14:55:23,237 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 7c8a55283e262bc196e8394e111484b6 in 1678ms, sequenceid=17, compaction requested=true 2023-02-12 14:55:23,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:23,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:23,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:23,237 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:23,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:23,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:23,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:23,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-12 14:55:23,237 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-12 14:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,239 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:23,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:23,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:23,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-12 14:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6cc26809[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:23,240 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-12 14:55:23,240 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,241 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,241 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-12 14:55:23,241 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-12 14:55:23,241 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:23,241 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,241 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:23,244 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,244 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-12 14:55:23,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:23,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:23,244 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:23,244 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-12 14:55:23,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:23,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:23,246 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-12 14:55:23,246 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:23,246 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:23,246 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-12 14:55:23,246 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-12 14:55:23,247 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:23,247 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:23,247 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:23,247 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:23,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:23,248 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:23,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:23,249 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:23,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:23,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:23,251 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:23,251 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink because node does not exist (not an error) 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:23,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:23,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-12 14:55:23,251 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:23,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:23,251 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:23,351 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:23,352 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:23,362 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-12 14:55:23,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676213723362 (current time:1676213723362). 2023-02-12 14:55:23,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-12 14:55:23,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] zookeeper.ReadOnlyZKClient(139): Connect 0x66bbf6ad to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:23,368 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@51f6dad9, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:23,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:23,375 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:57686, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:23,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x66bbf6ad to 127.0.0.1:61565 2023-02-12 14:55:23,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:23,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-12 14:55:23,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(648): Table enabled, starting distributed snapshots for { ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-12 14:55:23,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-12 14:55:23,385 DEBUG [PEWorker-3] locking.LockProcedure(309): LOCKED pid=86, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-12 14:55:23,385 INFO [PEWorker-3] procedure2.TimeoutExecutorThread(81): ADDED pid=86, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676214323385 2023-02-12 14:55:23,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(653): Started snapshot: { ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-12 14:55:23,386 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running FLUSH table snapshot snaptb-1676213723352 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-12 14:55:23,387 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=86, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-12 14:55:23,387 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure2.ProcedureExecutor(1029): Stored pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-12 14:55:23,388 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=86, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 3 msec 2023-02-12 14:55:23,388 DEBUG [PEWorker-5] locking.LockProcedure(309): LOCKED pid=87, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-12 14:55:23,388 INFO [PEWorker-5] procedure2.TimeoutExecutorThread(81): ADDED pid=87, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED; timeout=600000, timestamp=1676214323388 2023-02-12 14:55:23,390 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:23,390 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-12 14:55:23,490 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:23,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:23,494 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:23,494 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-12 14:55:23,694 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:23,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:23,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:23,695 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-12 14:55:23,800 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] procedure.ProcedureCoordinator(165): Submitting procedure snaptb-1676213723352 2023-02-12 14:55:23,801 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'snaptb-1676213723352' 2023-02-12 14:55:23,801 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-12 14:55:23,801 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'snaptb-1676213723352' starting 'acquire' 2023-02-12 14:55:23,801 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'snaptb-1676213723352', kicking off acquire phase on members. 2023-02-12 14:55:23,801 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:23,801 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/online-snapshot/acquired/snaptb-1676213723352 2023-02-12 14:55:23,803 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/online-snapshot/acquired/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-12 14:55:23,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-12 14:55:23,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-12 14:55:23,803 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/acquired/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,803 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:23,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/online-snapshot/acquired/snaptb-1676213723352 2023-02-12 14:55:23,803 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 68 2023-02-12 14:55:23,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/online-snapshot/acquired/snaptb-1676213723352 2023-02-12 14:55:23,804 DEBUG [zk-event-processor-pool-0] snapshot.RegionServerSnapshotManager(175): Launching subprocedure for snapshot snaptb-1676213723352 from table testCompactionWithHFileLink type FLUSH 2023-02-12 14:55:23,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:snaptb-1676213723352 2023-02-12 14:55:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'snaptb-1676213723352' with timeout 300000ms 2023-02-12 14:55:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 300000 ms 2023-02-12 14:55:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'snaptb-1676213723352' starting 'acquire' stage 2023-02-12 14:55:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'snaptb-1676213723352' locally acquired 2023-02-12 14:55:23,807 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (snaptb-1676213723352) in zk 2023-02-12 14:55:23,808 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:23,808 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/acquired/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-12 14:55:23,808 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:23,808 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'snaptb-1676213723352' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:23,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676213723352 2023-02-12 14:55:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:23,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'snaptb-1676213723352' on coordinator 2023-02-12 14:55:23,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@12ed4d02[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:23,809 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'snaptb-1676213723352' starting 'in-barrier' execution. 2023-02-12 14:55:23,809 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:23,811 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:23,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:23,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:23,811 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'snaptb-1676213723352' received 'reached' from coordinator. 2023-02-12 14:55:23,811 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/online-snapshot/reached/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:23,811 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:23,811 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:23,811 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:23,812 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] snapshot.FlushSnapshotSubprocedure(170): Flush Snapshot Tasks submitted for 3 regions 2023-02-12 14:55:23,812 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(97): Starting snapshot operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:23,812 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(301): Waiting for local region snapshots to finish. 2023-02-12 14:55:23,812 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. started... 2023-02-12 14:55:23,812 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. started... 2023-02-12 14:55:23,812 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(110): Flush Snapshotting region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. started... 2023-02-12 14:55:23,812 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:23,812 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:23,812 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:23,813 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.' region-info for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,813 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.' region-info for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,813 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(238): Storing 'testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.' region-info for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,818 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-12 14:55:23,818 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-12 14:55:23,818 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(243): Creating references for hfiles 2023-02-12 14:55:23,821 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3] hfiles 2023-02-12 14:55:23,821 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa] hfiles 2023-02-12 14:55:23,821 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68] hfiles 2023-02-12 14:55:23,821 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,821 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,821 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,969 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:55:23,969 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(142): Chore: CompactedHFilesCleaner missed its start time 2023-02-12 14:55:23,969 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x53bf1552 to 127.0.0.1:61565 2023-02-12 14:55:23,969 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:23,969 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,969 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,969 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,969 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,969 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b] hfiles 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3] hfiles 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(253): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e] hfiles 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (1/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (2/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (3/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,970 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.SnapshotManifest(262): Adding reference for file (4/4): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3 for snapshot=snaptb-1676213723352 2023-02-12 14:55:23,995 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:23,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:23,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:23,996 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-12 14:55:24,382 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. completed. 2023-02-12 14:55:24,382 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-1] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:24,382 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. completed. 2023-02-12 14:55:24,383 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-0] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:24,383 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 1/3 local region snapshots. 2023-02-12 14:55:24,383 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 2/3 local region snapshots. 2023-02-12 14:55:24,383 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(137): ... Flush Snapshotting region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. completed. 2023-02-12 14:55:24,383 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-snapshot-pool-2] snapshot.FlushSnapshotSubprocedure$RegionSnapshotTask(140): Closing snapshot operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:24,383 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(312): Completed 3/3 local region snapshots. 2023-02-12 14:55:24,383 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(314): Completed 3 local region snapshots. 2023-02-12 14:55:24,383 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] snapshot.RegionServerSnapshotManager$SnapshotSubprocedurePool(345): cancelling 0 tasks for snapshot jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,384 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'snaptb-1676213723352' locally completed 2023-02-12 14:55:24,384 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'snaptb-1676213723352' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:24,385 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'snaptb-1676213723352' has notified controller of completion 2023-02-12 14:55:24,385 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,385 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:24,385 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'snaptb-1676213723352' completed. 2023-02-12 14:55:24,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/reached/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:24,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-12 14:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676213723352 2023-02-12 14:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676213723352 2023-02-12 14:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'snaptb-1676213723352' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:24,387 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'snaptb-1676213723352', counting down latch. Waiting for 0 more 2023-02-12 14:55:24,387 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'snaptb-1676213723352' execution completed 2023-02-12 14:55:24,387 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:24,387 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:24,387 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:snaptb-1676213723352 2023-02-12 14:55:24,387 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure snaptb-1676213723352including nodes /hbase/online-snapshot/acquired /hbase/online-snapshot/reached /hbase/online-snapshot/abort 2023-02-12 14:55:24,389 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:24,389 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:24,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/online-snapshot 2023-02-12 14:55:24,389 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-12 14:55:24,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-12 14:55:24,389 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/acquired/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:24,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-12 14:55:24,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:24,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676213723352 2023-02-12 14:55:24,390 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:24,391 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/online-snapshot/reached/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676213723352 2023-02-12 14:55:24,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:24,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----snaptb-1676213723352 2023-02-12 14:55:24,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676213723352 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/acquired 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/acquired/snaptb-1676213723352 2023-02-12 14:55:24,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/online-snapshot/acquired 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676213723352/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:24,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/reached/snaptb-1676213723352 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/online-snapshot/abort/snaptb-1676213723352 2023-02-12 14:55:24,393 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(97): Done waiting - online snapshot for snaptb-1676213723352 2023-02-12 14:55:24,393 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/online-snapshot/abort 2023-02-12 14:55:24,393 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.EnabledTableSnapshotHandler(113): Taking snapshot for mob files in table testCompactionWithHFileLink 2023-02-12 14:55:24,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/online-snapshot/abort 2023-02-12 14:55:24,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:24,393 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(191): Storing mob region '{ENCODED => 0e7b6fa9e81bd62588edd7d77d6165e4, NAME => 'testCompactionWithHFileLink,.mob,0.0e7b6fa9e81bd62588edd7d77d6165e4.', STARTKEY => '.mob', ENDKEY => ''}' region-info for snapshot=snaptb-1676213723352 2023-02-12 14:55:24,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-12 14:55:24,393 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(196): Creating references for mob files 2023-02-12 14:55:24,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/online-snapshot/abort/snaptb-1676213723352 because node does not exist (not an error) 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8] mob files 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 2023-02-12 14:55:24,395 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 2023-02-12 14:55:24,396 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302123415c46c45d6470d9112000f8bf04c70, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302129b36e5cfbffa44e499012c03047dd37b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cc09253e7c7f4e1683b5d6f6775972fb, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212d4be49120391415bb2ea6fe55808e6b9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212283dbf899f7f49f1a537097648f293ad, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302125e1fe8592e164715b749d5d893383f7f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021277eedf1d464140ec88991fce862497b9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212af91ff60a0254fbfa2c8d9efcf4a313d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302120fb46ed2fc32453fbb3b6d6cec5237f6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302122cc3e41fa25843a1848eaeae94611883, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123748b931d9074f96b8c1f4dd73ecc85b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123f15fb6bd0f04990aba6745eee75a1c9] mob files 2023-02-12 14:55:24,396 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (1/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302123415c46c45d6470d9112000f8bf04c70 2023-02-12 14:55:24,396 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (2/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302129b36e5cfbffa44e499012c03047dd37b 2023-02-12 14:55:24,396 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (3/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cc09253e7c7f4e1683b5d6f6775972fb 2023-02-12 14:55:24,396 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (4/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212d4be49120391415bb2ea6fe55808e6b9 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (5/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212283dbf899f7f49f1a537097648f293ad 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (6/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302125e1fe8592e164715b749d5d893383f7f 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (7/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021277eedf1d464140ec88991fce862497b9 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (8/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212af91ff60a0254fbfa2c8d9efcf4a313d 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (9/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302120fb46ed2fc32453fbb3b6d6cec5237f6 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (10/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302122cc3e41fa25843a1848eaeae94611883 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (11/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123748b931d9074f96b8c1f4dd73ecc85b 2023-02-12 14:55:24,397 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(363): Adding reference for mob file (12/12): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123f15fb6bd0f04990aba6745eee75a1c9 2023-02-12 14:55:24,496 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:24,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:24,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:24,497 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-12 14:55:24,801 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for snaptb-1676213723352 2023-02-12 14:55:24,803 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/snaptb-1676213723352 2023-02-12 14:55:25,248 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/snaptb-1676213723352 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/snaptb-1676213723352 2023-02-12 14:55:25,498 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:25,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:25,498 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:25,498 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#6) Sleeping: 2000ms while waiting for snapshot completion. 2023-02-12 14:55:26,062 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot snaptb-1676213723352 of table testCompactionWithHFileLink completed 2023-02-12 14:55:26,062 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/snaptb-1676213723352 2023-02-12 14:55:26,062 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/snaptb-1676213723352 2023-02-12 14:55:26,062 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running FLUSH table snapshot snaptb-1676213723352 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676213723386Completed referencing HFiles for the mob region of table: testCompactionWithHFileLink at 1676213724801 (+1415 ms)Consolidate snapshot: snaptb-1676213723352 at 1676213724801Loading Region manifests for snaptb-1676213723352 at 1676213724802 (+1 ms)Writing data manifest for snaptb-1676213723352 at 1676213724810 (+8 ms)Verifying snapshot: snaptb-1676213723352 at 1676213725217 (+407 ms)Snapshot snaptb-1676213723352 of table testCompactionWithHFileLink completed at 1676213726062 (+845 ms) 2023-02-12 14:55:26,063 DEBUG [PEWorker-2] locking.LockProcedure(242): UNLOCKED pid=87, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED 2023-02-12 14:55:26,065 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=87, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=SHARED in 2.6760 sec 2023-02-12 14:55:27,499 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:27,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:27,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] snapshot.SnapshotManager(401): Snapshot '{ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-12 14:55:27,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:27,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testCompactionWithHFileLink was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:27,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testCompactionWithHFileLink 2023-02-12 14:55:27,528 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testCompactionWithHFileLink' 2023-02-12 14:55:27,528 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:27,528 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testCompactionWithHFileLink' starting 'acquire' 2023-02-12 14:55:27,528 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testCompactionWithHFileLink', kicking off acquire phase on members. 2023-02-12 14:55:27,529 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:27,529 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:27,531 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:27,531 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:27,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:27,532 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:27,532 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:27,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:27,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testCompactionWithHFileLink 2023-02-12 14:55:27,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testCompactionWithHFileLink 2023-02-12 14:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testCompactionWithHFileLink' with timeout 60000ms 2023-02-12 14:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testCompactionWithHFileLink' starting 'acquire' stage 2023-02-12 14:55:27,533 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:27,533 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:27,533 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:27,533 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:27,533 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. started... 2023-02-12 14:55:27,533 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. started... 2023-02-12 14:55:27,533 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. started... 2023-02-12 14:55:27,533 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 73ab03d290bf433ec02678876bd209ea 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:55:27,533 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c330f294c6804231199503c93a0f53d8 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:55:27,533 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7c8a55283e262bc196e8394e111484b6 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:55:27,948 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/02485cf7c4404ef8979d28a95c059a78 2023-02-12 14:55:27,954 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02485cf7c4404ef8979d28a95c059a78 2023-02-12 14:55:28,348 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/63f29f1afb0e454187d9e5a2aef9f9f4 2023-02-12 14:55:28,352 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63f29f1afb0e454187d9e5a2aef9f9f4 2023-02-12 14:55:28,352 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/97187333769d4100aeb3d55efb790232 2023-02-12 14:55:28,355 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97187333769d4100aeb3d55efb790232 2023-02-12 14:55:28,767 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/5b4ad8b24a7e481b9265b82391f604d4 2023-02-12 14:55:28,771 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b4ad8b24a7e481b9265b82391f604d4 2023-02-12 14:55:28,771 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/02485cf7c4404ef8979d28a95c059a78 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/02485cf7c4404ef8979d28a95c059a78 2023-02-12 14:55:28,773 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/f89471a9746848c89135e34618cef57c 2023-02-12 14:55:28,775 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 02485cf7c4404ef8979d28a95c059a78 2023-02-12 14:55:28,775 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/02485cf7c4404ef8979d28a95c059a78, entries=3, sequenceid=24, filesize=5.0 K 2023-02-12 14:55:28,776 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/5b4ad8b24a7e481b9265b82391f604d4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/5b4ad8b24a7e481b9265b82391f604d4 2023-02-12 14:55:28,776 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f89471a9746848c89135e34618cef57c 2023-02-12 14:55:28,777 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/97187333769d4100aeb3d55efb790232 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/97187333769d4100aeb3d55efb790232 2023-02-12 14:55:28,779 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b4ad8b24a7e481b9265b82391f604d4 2023-02-12 14:55:28,779 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/5b4ad8b24a7e481b9265b82391f604d4, entries=1, sequenceid=24, filesize=5.0 K 2023-02-12 14:55:28,779 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c330f294c6804231199503c93a0f53d8 in 1246ms, sequenceid=24, compaction requested=true 2023-02-12 14:55:28,780 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:28,780 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:28,780 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:28,780 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 97187333769d4100aeb3d55efb790232 2023-02-12 14:55:28,780 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/97187333769d4100aeb3d55efb790232, entries=3, sequenceid=24, filesize=5.0 K 2023-02-12 14:55:28,781 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/f89471a9746848c89135e34618cef57c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f89471a9746848c89135e34618cef57c 2023-02-12 14:55:28,784 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f89471a9746848c89135e34618cef57c 2023-02-12 14:55:28,784 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f89471a9746848c89135e34618cef57c, entries=1, sequenceid=24, filesize=5.0 K 2023-02-12 14:55:28,785 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 73ab03d290bf433ec02678876bd209ea in 1252ms, sequenceid=24, compaction requested=true 2023-02-12 14:55:28,785 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:28,785 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:28,785 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:29,163 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=24, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/d73ccc3770074f4e9410ba95567ffd6c 2023-02-12 14:55:29,167 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d73ccc3770074f4e9410ba95567ffd6c 2023-02-12 14:55:29,168 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/63f29f1afb0e454187d9e5a2aef9f9f4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63f29f1afb0e454187d9e5a2aef9f9f4 2023-02-12 14:55:29,170 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63f29f1afb0e454187d9e5a2aef9f9f4 2023-02-12 14:55:29,171 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63f29f1afb0e454187d9e5a2aef9f9f4, entries=3, sequenceid=24, filesize=5.0 K 2023-02-12 14:55:29,171 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/d73ccc3770074f4e9410ba95567ffd6c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/d73ccc3770074f4e9410ba95567ffd6c 2023-02-12 14:55:29,174 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d73ccc3770074f4e9410ba95567ffd6c 2023-02-12 14:55:29,174 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/d73ccc3770074f4e9410ba95567ffd6c, entries=1, sequenceid=24, filesize=5.0 K 2023-02-12 14:55:29,175 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 7c8a55283e262bc196e8394e111484b6 in 1642ms, sequenceid=24, compaction requested=true 2023-02-12 14:55:29,175 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:29,175 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:29,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:29,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:29,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testCompactionWithHFileLink' locally acquired 2023-02-12 14:55:29,175 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testCompactionWithHFileLink) in zk 2023-02-12 14:55:29,177 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:29,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:29,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:29,177 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testCompactionWithHFileLink' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:29,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:29,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:29,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:29,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testCompactionWithHFileLink' on coordinator 2023-02-12 14:55:29,178 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testCompactionWithHFileLink' starting 'in-barrier' execution. 2023-02-12 14:55:29,178 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cadf3e[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:29,178 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,182 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,183 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testCompactionWithHFileLink' received 'reached' from coordinator. 2023-02-12 14:55:29,183 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,183 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:29,183 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testCompactionWithHFileLink' locally completed 2023-02-12 14:55:29,183 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testCompactionWithHFileLink' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:29,184 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testCompactionWithHFileLink' has notified controller of completion 2023-02-12 14:55:29,184 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:29,184 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,184 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testCompactionWithHFileLink' completed. 2023-02-12 14:55:29,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:29,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:29,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:29,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:29,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:29,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:29,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:29,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testCompactionWithHFileLink' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:29,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testCompactionWithHFileLink', counting down latch. Waiting for 0 more 2023-02-12 14:55:29,186 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testCompactionWithHFileLink' execution completed 2023-02-12 14:55:29,186 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:29,186 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:29,186 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testCompactionWithHFileLink 2023-02-12 14:55:29,186 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testCompactionWithHFileLinkincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:29,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:29,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:29,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:29,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:29,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:29,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:29,187 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:29,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:29,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:29,187 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:29,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:29,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:29,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:29,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:29,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:29,189 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:29,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testCompactionWithHFileLink 2023-02-12 14:55:29,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:29,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:29,191 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:29,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:29,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:29,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testCompactionWithHFileLink' 2023-02-12 14:55:29,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:29,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testCompactionWithHFileLink 2023-02-12 14:55:29,191 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:29,191 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:29,192 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,192 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testCompactionWithHFileLink 2023-02-12 14:55:29,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:29,192 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testCompactionWithHFileLink'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:29,192 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testCompactionWithHFileLink 2023-02-12 14:55:29,192 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:29,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:29,292 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:29,293 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:29,293 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:29,293 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 7c8a55283e262bc196e8394e111484b6/family1 is initiating major compaction (all files) 2023-02-12 14:55:29,294 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:29,294 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:29,294 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 7c8a55283e262bc196e8394e111484b6/family1 in testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:29,295 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63f29f1afb0e454187d9e5a2aef9f9f4] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp, totalSize=26.7 K 2023-02-12 14:55:29,295 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting ecdc577f07da4cb1ad7ba1378a36e7d2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716182 2023-02-12 14:55:29,295 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 5c12eba08d1240b9b5042b98c3072cf6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:29,296 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 5ba1a1c8bb5046099d4c27b359ce7176, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:29,296 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 6f3d54677c014906aa0b5cb62516a2c3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:29,296 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 63f29f1afb0e454187d9e5a2aef9f9f4, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-12 14:55:29,309 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 7c8a55283e262bc196e8394e111484b6#family1#compaction#107 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:29,720 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230212307c506b801749649dad3690e606a584_del 2023-02-12 14:55:29,720 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212307c506b801749649dad3690e606a584_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212307c506b801749649dad3690e606a584_del 2023-02-12 14:55:30,533 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c952dd9230b44b288a115e9b4b73f5b3 2023-02-12 14:55:30,534 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/c952dd9230b44b288a115e9b4b73f5b3 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/c952dd9230b44b288a115e9b4b73f5b3 2023-02-12 14:55:30,537 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c952dd9230b44b288a115e9b4b73f5b3 2023-02-12 14:55:30,538 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7c8a55283e262bc196e8394e111484b6/family1 of 7c8a55283e262bc196e8394e111484b6 into c952dd9230b44b288a115e9b4b73f5b3(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:30,538 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:30,538 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:30,538 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 7c8a55283e262bc196e8394e111484b6/family2 is initiating major compaction (all files) 2023-02-12 14:55:30,538 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:30,538 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:30,539 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 7c8a55283e262bc196e8394e111484b6/family2 in testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:30,539 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/d73ccc3770074f4e9410ba95567ffd6c] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp, totalSize=25.4 K 2023-02-12 14:55:30,540 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 39c69ce2cdc04e89a1b8fcc62e0e41b2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716182 2023-02-12 14:55:30,540 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting a235bba083664fa998b43af265d0eed2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:30,540 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 30098feffd2e47808617c86382a508a5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:30,541 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 053d84dda82248ecaeb1d4d178f7710e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:30,541 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting d73ccc3770074f4e9410ba95567ffd6c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-12 14:55:30,553 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 7c8a55283e262bc196e8394e111484b6#family2#compaction#108 average throughput is 0.35 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:30,571 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63f29f1afb0e454187d9e5a2aef9f9f4] to archive 2023-02-12 14:55:30,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:30,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/ecdc577f07da4cb1ad7ba1378a36e7d2 2023-02-12 14:55:30,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5c12eba08d1240b9b5042b98c3072cf6 2023-02-12 14:55:30,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/5ba1a1c8bb5046099d4c27b359ce7176 2023-02-12 14:55:30,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/6f3d54677c014906aa0b5cb62516a2c3 2023-02-12 14:55:30,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63f29f1afb0e454187d9e5a2aef9f9f4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63f29f1afb0e454187d9e5a2aef9f9f4 2023-02-12 14:55:30,961 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230212cfe43cb34e8e4fb38507a4f4f3d42c4b_del 2023-02-12 14:55:30,961 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212cfe43cb34e8e4fb38507a4f4f3d42c4b_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cfe43cb34e8e4fb38507a4f4f3d42c4b_del 2023-02-12 14:55:31,774 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26b37568688f46b585aec1f729c5fcdd 2023-02-12 14:55:31,774 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/26b37568688f46b585aec1f729c5fcdd as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/26b37568688f46b585aec1f729c5fcdd 2023-02-12 14:55:31,778 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 26b37568688f46b585aec1f729c5fcdd 2023-02-12 14:55:31,779 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 7c8a55283e262bc196e8394e111484b6/family2 of 7c8a55283e262bc196e8394e111484b6 into 26b37568688f46b585aec1f729c5fcdd(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:31,779 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:31,779 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:31,779 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 73ab03d290bf433ec02678876bd209ea/family1 is initiating major compaction (all files) 2023-02-12 14:55:31,779 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:31,779 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:31,779 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 73ab03d290bf433ec02678876bd209ea/family1 in testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:31,779 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/97187333769d4100aeb3d55efb790232] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp, totalSize=26.7 K 2023-02-12 14:55:31,780 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting e03728432e5947eda80697c61f891d10, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716183 2023-02-12 14:55:31,780 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 49707ccb4e4f44f28a5e46acdf4a01f4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:31,780 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting a0644edaa9354d88938f406a3dbc2943, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:31,781 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 89aadb319ba44c88a9214d1ad6836b68, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:31,781 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 97187333769d4100aeb3d55efb790232, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-12 14:55:31,790 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 73ab03d290bf433ec02678876bd209ea#family1#compaction#109 average throughput is 0.85 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:31,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/d73ccc3770074f4e9410ba95567ffd6c] to archive 2023-02-12 14:55:31,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:31,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/39c69ce2cdc04e89a1b8fcc62e0e41b2 2023-02-12 14:55:31,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/a235bba083664fa998b43af265d0eed2 2023-02-12 14:55:31,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/30098feffd2e47808617c86382a508a5 2023-02-12 14:55:31,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/053d84dda82248ecaeb1d4d178f7710e 2023-02-12 14:55:31,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/d73ccc3770074f4e9410ba95567ffd6c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/d73ccc3770074f4e9410ba95567ffd6c 2023-02-12 14:55:32,197 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212482234933f8f42a6b53846a30766c0cb_del 2023-02-12 14:55:32,197 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212482234933f8f42a6b53846a30766c0cb_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212482234933f8f42a6b53846a30766c0cb_del 2023-02-12 14:55:33,010 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b3f47004a60c47d7a21a470249efb1d0 2023-02-12 14:55:33,011 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/b3f47004a60c47d7a21a470249efb1d0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/b3f47004a60c47d7a21a470249efb1d0 2023-02-12 14:55:33,014 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b3f47004a60c47d7a21a470249efb1d0 2023-02-12 14:55:33,015 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 73ab03d290bf433ec02678876bd209ea/family1 of 73ab03d290bf433ec02678876bd209ea into b3f47004a60c47d7a21a470249efb1d0(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:33,015 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:33,015 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:33,015 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 73ab03d290bf433ec02678876bd209ea/family2 is initiating major compaction (all files) 2023-02-12 14:55:33,015 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:33,015 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:33,015 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 73ab03d290bf433ec02678876bd209ea/family2 in testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:33,015 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f89471a9746848c89135e34618cef57c] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp, totalSize=25.4 K 2023-02-12 14:55:33,016 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f2a416bc30b44a53874f33431ceee564, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716183 2023-02-12 14:55:33,016 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 20f9cf39b0594acf9ec77c570d62698c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:33,016 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f31845a05242422fb2c55797c8de37b4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:33,017 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 27b7d09543864486b8af9b45fb33b2f3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:33,017 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f89471a9746848c89135e34618cef57c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-12 14:55:33,026 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 73ab03d290bf433ec02678876bd209ea#family2#compaction#110 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:33,069 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/97187333769d4100aeb3d55efb790232] to archive 2023-02-12 14:55:33,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:33,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/e03728432e5947eda80697c61f891d10 2023-02-12 14:55:33,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/49707ccb4e4f44f28a5e46acdf4a01f4 2023-02-12 14:55:33,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/a0644edaa9354d88938f406a3dbc2943 2023-02-12 14:55:33,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/89aadb319ba44c88a9214d1ad6836b68 2023-02-12 14:55:33,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/97187333769d4100aeb3d55efb790232 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/97187333769d4100aeb3d55efb790232 2023-02-12 14:55:33,433 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212e9e03c7faa9b493b8ba7a95e7b971683_del 2023-02-12 14:55:33,433 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212e9e03c7faa9b493b8ba7a95e7b971683_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212e9e03c7faa9b493b8ba7a95e7b971683_del 2023-02-12 14:55:34,247 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2aa0989bffb34d159ed605901e7d0b39 2023-02-12 14:55:34,248 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/2aa0989bffb34d159ed605901e7d0b39 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/2aa0989bffb34d159ed605901e7d0b39 2023-02-12 14:55:34,251 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2aa0989bffb34d159ed605901e7d0b39 2023-02-12 14:55:34,252 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 73ab03d290bf433ec02678876bd209ea/family2 of 73ab03d290bf433ec02678876bd209ea into 2aa0989bffb34d159ed605901e7d0b39(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:34,252 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:34,252 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:34,252 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): c330f294c6804231199503c93a0f53d8/family1 is initiating major compaction (all files) 2023-02-12 14:55:34,253 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:34,253 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:34,253 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of c330f294c6804231199503c93a0f53d8/family1 in testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:34,253 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/02485cf7c4404ef8979d28a95c059a78] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp, totalSize=26.7 K 2023-02-12 14:55:34,253 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 69f858874fa141889a3f3d14d75bc3fd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716181 2023-02-12 14:55:34,253 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 41be1e3dff234cadbea7808c2c672e0c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717967 2023-02-12 14:55:34,254 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 526d1d45234641e5877c28f0ee1c4fe6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719756 2023-02-12 14:55:34,254 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 3e7296de83954248a5d807c9f40e18fa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:34,254 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 02485cf7c4404ef8979d28a95c059a78, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-12 14:55:34,264 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): c330f294c6804231199503c93a0f53d8#family1#compaction#111 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:34,269 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f89471a9746848c89135e34618cef57c] to archive 2023-02-12 14:55:34,270 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:34,271 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f2a416bc30b44a53874f33431ceee564 2023-02-12 14:55:34,272 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/20f9cf39b0594acf9ec77c570d62698c 2023-02-12 14:55:34,273 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f31845a05242422fb2c55797c8de37b4 2023-02-12 14:55:34,274 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/27b7d09543864486b8af9b45fb33b2f3 2023-02-12 14:55:34,275 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f89471a9746848c89135e34618cef57c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/f89471a9746848c89135e34618cef57c 2023-02-12 14:55:34,673 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021218a0c0ddd22f42238416a2e07ce032e5_del 2023-02-12 14:55:34,673 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021218a0c0ddd22f42238416a2e07ce032e5_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021218a0c0ddd22f42238416a2e07ce032e5_del 2023-02-12 14:55:35,487 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8efb885f124e47b29197be1ea0bf9cec 2023-02-12 14:55:35,487 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/8efb885f124e47b29197be1ea0bf9cec as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/8efb885f124e47b29197be1ea0bf9cec 2023-02-12 14:55:35,491 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8efb885f124e47b29197be1ea0bf9cec 2023-02-12 14:55:35,492 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c330f294c6804231199503c93a0f53d8/family1 of c330f294c6804231199503c93a0f53d8 into 8efb885f124e47b29197be1ea0bf9cec(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:35,492 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:35,492 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:35,492 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): c330f294c6804231199503c93a0f53d8/family2 is initiating major compaction (all files) 2023-02-12 14:55:35,492 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:35,492 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:35,492 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of c330f294c6804231199503c93a0f53d8/family2 in testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:35,492 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/5b4ad8b24a7e481b9265b82391f604d4] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp, totalSize=25.4 K 2023-02-12 14:55:35,493 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 18484b7a53624894893277beb20e6d58, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716181 2023-02-12 14:55:35,493 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 700fda40ac8649ef9a44f4cfdc556a9f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717967 2023-02-12 14:55:35,493 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 4aba417dd6d142129901d792ec2f7631, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719756 2023-02-12 14:55:35,494 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 66e244f61f03457384884137acf1b58b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:35,494 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 5b4ad8b24a7e481b9265b82391f604d4, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=24, earliestPutTs=9223372036854775807 2023-02-12 14:55:35,504 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): c330f294c6804231199503c93a0f53d8#family2#compaction#112 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:35,570 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/02485cf7c4404ef8979d28a95c059a78] to archive 2023-02-12 14:55:35,571 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:35,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/69f858874fa141889a3f3d14d75bc3fd 2023-02-12 14:55:35,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/41be1e3dff234cadbea7808c2c672e0c 2023-02-12 14:55:35,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/526d1d45234641e5877c28f0ee1c4fe6 2023-02-12 14:55:35,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/3e7296de83954248a5d807c9f40e18fa 2023-02-12 14:55:35,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/02485cf7c4404ef8979d28a95c059a78 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/02485cf7c4404ef8979d28a95c059a78 2023-02-12 14:55:35,912 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230212ae32d3fec30348d69aed4b465836169d_del 2023-02-12 14:55:35,912 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212ae32d3fec30348d69aed4b465836169d_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230212ae32d3fec30348d69aed4b465836169d_del 2023-02-12 14:55:36,726 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e4db3ffeb224d03a93d1e88f994b790 2023-02-12 14:55:36,727 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/2e4db3ffeb224d03a93d1e88f994b790 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/2e4db3ffeb224d03a93d1e88f994b790 2023-02-12 14:55:36,730 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2e4db3ffeb224d03a93d1e88f994b790 2023-02-12 14:55:36,731 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c330f294c6804231199503c93a0f53d8/family2 of c330f294c6804231199503c93a0f53d8 into 2e4db3ffeb224d03a93d1e88f994b790(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:36,731 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:36,736 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:36,740 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:36,746 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:36,747 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:36,756 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-12 14:55:36,770 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/5b4ad8b24a7e481b9265b82391f604d4] to archive 2023-02-12 14:55:36,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:36,773 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/18484b7a53624894893277beb20e6d58 2023-02-12 14:55:36,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/700fda40ac8649ef9a44f4cfdc556a9f 2023-02-12 14:55:36,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/4aba417dd6d142129901d792ec2f7631 2023-02-12 14:55:36,777 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/66e244f61f03457384884137acf1b58b 2023-02-12 14:55:36,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/5b4ad8b24a7e481b9265b82391f604d4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/5b4ad8b24a7e481b9265b82391f604d4 2023-02-12 14:55:36,795 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-12 14:55:36,798 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e2023021218a0c0ddd22f42238416a2e07ce032e5_del 2023-02-12 14:55:36,800 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230212307c506b801749649dad3690e606a584_del 2023-02-12 14:55:36,803 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212482234933f8f42a6b53846a30766c0cb_del 2023-02-12 14:55:36,803 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:36,803 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(139): Connect 0x314f3d9c to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:36,808 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2a8e9e10, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:36,808 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:36,808 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:36,808 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:37,263 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302127078e27556344a0aa5c4c69746961093 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302127078e27556344a0aa5c4c69746961093 2023-02-12 14:55:37,263 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212a619aeb36ff447dc977e0f5ccec0a359 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212a619aeb36ff447dc977e0f5ccec0a359 2023-02-12 14:55:37,264 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:37,264 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:37,277 DEBUG [MobFileCompactionChore-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:37,281 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39838, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:37,286 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:55:37,288 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60708, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:55:37,297 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/86ea517184fc496eb748760c289b8b59 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-12 14:55:37,297 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/8eed1de5ac4b451ea7ad08c08088e0b0 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-12 14:55:37,299 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/86ea517184fc496eb748760c289b8b59}] 2023-02-12 14:55:37,299 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/8eed1de5ac4b451ea7ad08c08088e0b0}] 2023-02-12 14:55:37,303 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/8eed1de5ac4b451ea7ad08c08088e0b0 for inclusion in c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:37,303 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/86ea517184fc496eb748760c289b8b59 for inclusion in 73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:37,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-12 14:55:37,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-12 14:55:37,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last=1 2023-02-12 14:55:37,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-12 14:55:37,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:37,305 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:37,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/8eed1de5ac4b451ea7ad08c08088e0b0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__alifdddcj6gf3hrrd8d8rolhsqgue7pml8mp0ka1ue84i3hq814guu9r618n6ggc/family1/8eed1de5ac4b451ea7ad08c08088e0b0 2023-02-12 14:55:37,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/86ea517184fc496eb748760c289b8b59 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__p8dc5qkt6ps9qjbk043slmri1e0ksuo3ckpq2g8nrc62fth1ee5pkoogdrockj31/family1/86ea517184fc496eb748760c289b8b59 2023-02-12 14:55:37,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__alifdddcj6gf3hrrd8d8rolhsqgue7pml8mp0ka1ue84i3hq814guu9r618n6ggc/family1/8eed1de5ac4b451ea7ad08c08088e0b0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_ 2023-02-12 14:55:37,308 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__p8dc5qkt6ps9qjbk043slmri1e0ksuo3ckpq2g8nrc62fth1ee5pkoogdrockj31/family1/86ea517184fc496eb748760c289b8b59 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_ 2023-02-12 14:55:37,309 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__alifdddcj6gf3hrrd8d8rolhsqgue7pml8mp0ka1ue84i3hq814guu9r618n6ggc/family1/8eed1de5ac4b451ea7ad08c08088e0b0 into c330f294c6804231199503c93a0f53d8/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_ - updating store file list. 2023-02-12 14:55:37,309 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__p8dc5qkt6ps9qjbk043slmri1e0ksuo3ckpq2g8nrc62fth1ee5pkoogdrockj31/family1/86ea517184fc496eb748760c289b8b59 into 73ab03d290bf433ec02678876bd209ea/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_ - updating store file list. 2023-02-12 14:55:37,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_ into c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:37,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__alifdddcj6gf3hrrd8d8rolhsqgue7pml8mp0ka1ue84i3hq814guu9r618n6ggc/family1/8eed1de5ac4b451ea7ad08c08088e0b0 into c330f294c6804231199503c93a0f53d8/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_) 2023-02-12 14:55:37,312 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_ into 73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:37,312 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__p8dc5qkt6ps9qjbk043slmri1e0ksuo3ckpq2g8nrc62fth1ee5pkoogdrockj31/family1/86ea517184fc496eb748760c289b8b59 into 73ab03d290bf433ec02678876bd209ea/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_) 2023-02-12 14:55:37,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__alifdddcj6gf3hrrd8d8rolhsqgue7pml8mp0ka1ue84i3hq814guu9r618n6ggc/family1/8eed1de5ac4b451ea7ad08c08088e0b0 2023-02-12 14:55:37,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__p8dc5qkt6ps9qjbk043slmri1e0ksuo3ckpq2g8nrc62fth1ee5pkoogdrockj31/family1/86ea517184fc496eb748760c289b8b59 2023-02-12 14:55:37,315 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:37,315 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:37,315 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:37,316 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:37,317 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 2023-02-12 14:55:37,317 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 2023-02-12 14:55:37,318 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 2023-02-12 14:55:37,318 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 2023-02-12 14:55:37,319 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 2023-02-12 14:55:37,319 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc 2023-02-12 14:55:37,320 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 2023-02-12 14:55:37,320 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-12 14:55:37,320 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212' 2023-02-12 14:55:37,320 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff 2023-02-12 14:55:37,320 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-12 14:55:37,320 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212' 2023-02-12 14:55:37,321 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:37,321 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:37,654 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212e4fc573a4b9d4ceeb7d6887e3c490581 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212e4fc573a4b9d4ceeb7d6887e3c490581 2023-02-12 14:55:37,654 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:37,678 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/d2d9d9d15fde4ecba3094486a6b04b14 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-12 14:55:37,678 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/d2d9d9d15fde4ecba3094486a6b04b14}] 2023-02-12 14:55:37,680 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/d2d9d9d15fde4ecba3094486a6b04b14 for inclusion in 7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:37,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-12 14:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-12 14:55:37,683 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:37,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/d2d9d9d15fde4ecba3094486a6b04b14 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__ibu7lnc8ltuqabmbqq8crvsrh9lsqemc1giap60pq900t46polagaurc7cgmp2b3/family1/d2d9d9d15fde4ecba3094486a6b04b14 2023-02-12 14:55:37,687 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__ibu7lnc8ltuqabmbqq8crvsrh9lsqemc1giap60pq900t46polagaurc7cgmp2b3/family1/d2d9d9d15fde4ecba3094486a6b04b14 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_ 2023-02-12 14:55:37,687 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__ibu7lnc8ltuqabmbqq8crvsrh9lsqemc1giap60pq900t46polagaurc7cgmp2b3/family1/d2d9d9d15fde4ecba3094486a6b04b14 into 7c8a55283e262bc196e8394e111484b6/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_ - updating store file list. 2023-02-12 14:55:37,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_ into 7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:37,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__ibu7lnc8ltuqabmbqq8crvsrh9lsqemc1giap60pq900t46polagaurc7cgmp2b3/family1/d2d9d9d15fde4ecba3094486a6b04b14 into 7c8a55283e262bc196e8394e111484b6/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_) 2023-02-12 14:55:37,690 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__ibu7lnc8ltuqabmbqq8crvsrh9lsqemc1giap60pq900t46polagaurc7cgmp2b3/family1/d2d9d9d15fde4ecba3094486a6b04b14 2023-02-12 14:55:37,692 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:37,693 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:37,694 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 2023-02-12 14:55:37,695 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b 2023-02-12 14:55:37,697 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf 2023-02-12 14:55:37,698 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 2023-02-12 14:55:37,698 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-12 14:55:37,698 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212' 2023-02-12 14:55:37,698 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:37,698 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:37,699 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:37,699 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021218a0c0ddd22f42238416a2e07ce032e5_del] 2023-02-12 14:55:37,699 DEBUG [Listener at localhost/37945] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:37,701 DEBUG [Listener at localhost/37945] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021218a0c0ddd22f42238416a2e07ce032e5_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e2023021218a0c0ddd22f42238416a2e07ce032e5_del 2023-02-12 14:55:37,701 INFO [Listener at localhost/37945] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-12 14:55:37,701 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212307c506b801749649dad3690e606a584_del] 2023-02-12 14:55:37,701 DEBUG [Listener at localhost/37945] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:37,702 DEBUG [Listener at localhost/37945] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212307c506b801749649dad3690e606a584_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212307c506b801749649dad3690e606a584_del 2023-02-12 14:55:37,703 INFO [Listener at localhost/37945] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-12 14:55:37,703 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212482234933f8f42a6b53846a30766c0cb_del] 2023-02-12 14:55:37,703 DEBUG [Listener at localhost/37945] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:37,704 DEBUG [Listener at localhost/37945] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212482234933f8f42a6b53846a30766c0cb_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c20230212482234933f8f42a6b53846a30766c0cb_del 2023-02-12 14:55:37,705 INFO [Listener at localhost/37945] mob.MobUtils(508): Table testCompactionWithHFileLink 1 expired mob files are deleted 2023-02-12 14:55:37,707 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:37,709 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:37,712 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:37,713 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:37,720 INFO [Listener at localhost/37945] client.HBaseAdmin$15(890): Started disable of testCompactionWithHFileLink 2023-02-12 14:55:37,720 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.HMaster$11(2405): Client=jenkins//172.31.14.131 disable testCompactionWithHFileLink 2023-02-12 14:55:37,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testCompactionWithHFileLink 2023-02-12 14:55:37,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 14:55:37,723 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213737723"}]},"ts":"1676213737723"} 2023-02-12 14:55:37,724 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLING in hbase:meta 2023-02-12 14:55:37,726 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testCompactionWithHFileLink to state=DISABLING 2023-02-12 14:55:37,726 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, UNASSIGN}, {pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, UNASSIGN}, {pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, UNASSIGN}] 2023-02-12 14:55:37,728 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, UNASSIGN 2023-02-12 14:55:37,728 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=91, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, UNASSIGN 2023-02-12 14:55:37,728 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=89, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, UNASSIGN 2023-02-12 14:55:37,728 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7c8a55283e262bc196e8394e111484b6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:37,729 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=73ab03d290bf433ec02678876bd209ea, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:37,729 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=c330f294c6804231199503c93a0f53d8, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:37,729 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213737728"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213737728"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213737728"}]},"ts":"1676213737728"} 2023-02-12 14:55:37,729 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213737728"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213737728"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213737728"}]},"ts":"1676213737728"} 2023-02-12 14:55:37,729 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213737729"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213737729"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213737729"}]},"ts":"1676213737729"} 2023-02-12 14:55:37,730 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=91, state=RUNNABLE; CloseRegionProcedure 73ab03d290bf433ec02678876bd209ea, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:37,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=90, state=RUNNABLE; CloseRegionProcedure 7c8a55283e262bc196e8394e111484b6, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:37,731 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=89, state=RUNNABLE; CloseRegionProcedure c330f294c6804231199503c93a0f53d8, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:37,824 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 14:55:37,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:37,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 73ab03d290bf433ec02678876bd209ea, disabling compactions & flushes 2023-02-12 14:55:37,882 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:37,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:37,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. after waiting 0 ms 2023-02-12 14:55:37,882 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:37,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-12 14:55:37,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:37,887 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 73ab03d290bf433ec02678876bd209ea: Waiting for close lock at 1676213737882Running coprocessor pre-close hooks at 1676213737882Disabling compacts and flushes for region at 1676213737882Disabling writes for close at 1676213737882Writing region close event to WAL at 1676213737883 (+1 ms)Running coprocessor post-close hooks at 1676213737886 (+3 ms)Closed at 1676213737887 (+1 ms) 2023-02-12 14:55:37,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:37,888 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:37,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c330f294c6804231199503c93a0f53d8, disabling compactions & flushes 2023-02-12 14:55:37,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:37,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:37,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. after waiting 0 ms 2023-02-12 14:55:37,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:37,889 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=91 updating hbase:meta row=73ab03d290bf433ec02678876bd209ea, regionState=CLOSED 2023-02-12 14:55:37,889 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213737889"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213737889"}]},"ts":"1676213737889"} 2023-02-12 14:55:37,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=91 2023-02-12 14:55:37,892 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=91, state=SUCCESS; CloseRegionProcedure 73ab03d290bf433ec02678876bd209ea, server=jenkins-hbase4.apache.org,45319,1676213617001 in 161 msec 2023-02-12 14:55:37,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-12 14:55:37,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, UNASSIGN in 166 msec 2023-02-12 14:55:37,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:37,895 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c330f294c6804231199503c93a0f53d8: Waiting for close lock at 1676213737889Running coprocessor pre-close hooks at 1676213737889Disabling compacts and flushes for region at 1676213737889Disabling writes for close at 1676213737889Writing region close event to WAL at 1676213737890 (+1 ms)Running coprocessor post-close hooks at 1676213737894 (+4 ms)Closed at 1676213737894 2023-02-12 14:55:37,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:37,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(109): Close 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:37,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7c8a55283e262bc196e8394e111484b6, disabling compactions & flushes 2023-02-12 14:55:37,896 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:37,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:37,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. after waiting 0 ms 2023-02-12 14:55:37,896 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:37,896 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=89 updating hbase:meta row=c330f294c6804231199503c93a0f53d8, regionState=CLOSED 2023-02-12 14:55:37,896 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213737896"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213737896"}]},"ts":"1676213737896"} 2023-02-12 14:55:37,900 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=89 2023-02-12 14:55:37,900 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=89, state=SUCCESS; CloseRegionProcedure c330f294c6804231199503c93a0f53d8, server=jenkins-hbase4.apache.org,45319,1676213617001 in 167 msec 2023-02-12 14:55:37,901 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, UNASSIGN in 174 msec 2023-02-12 14:55:37,902 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/recovered.edits/31.seqid, newMaxSeqId=31, maxSeqId=1 2023-02-12 14:55:37,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:37,903 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7c8a55283e262bc196e8394e111484b6: Waiting for close lock at 1676213737896Running coprocessor pre-close hooks at 1676213737896Disabling compacts and flushes for region at 1676213737896Disabling writes for close at 1676213737896Writing region close event to WAL at 1676213737899 (+3 ms)Running coprocessor post-close hooks at 1676213737903 (+4 ms)Closed at 1676213737903 2023-02-12 14:55:37,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(147): Closed 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:37,905 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=7c8a55283e262bc196e8394e111484b6, regionState=CLOSED 2023-02-12 14:55:37,905 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213737905"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213737905"}]},"ts":"1676213737905"} 2023-02-12 14:55:37,907 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=90 2023-02-12 14:55:37,907 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=90, state=SUCCESS; CloseRegionProcedure 7c8a55283e262bc196e8394e111484b6, server=jenkins-hbase4.apache.org,45319,1676213617001 in 176 msec 2023-02-12 14:55:37,908 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=88 2023-02-12 14:55:37,908 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, UNASSIGN in 181 msec 2023-02-12 14:55:37,909 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213737909"}]},"ts":"1676213737909"} 2023-02-12 14:55:37,910 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=DISABLED in hbase:meta 2023-02-12 14:55:37,911 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testCompactionWithHFileLink to state=DISABLED 2023-02-12 14:55:37,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; DisableTableProcedure table=testCompactionWithHFileLink in 192 msec 2023-02-12 14:55:38,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-12 14:55:38,025 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testCompactionWithHFileLink, procId: 88 completed 2023-02-12 14:55:38,035 INFO [Listener at localhost/37945] client.HBaseAdmin(2586): Taking restore-failsafe snapshot: hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:38,035 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1580): Client=jenkins//172.31.14.131 snapshot request for:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-12 14:55:38,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotDescriptionUtils(316): Creation time not specified, setting to:1676213738036 (current time:1676213738036). 2023-02-12 14:55:38,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotDescriptionUtils(332): Snapshot current TTL value: 0 resetting it to default value: 0 2023-02-12 14:55:38,036 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] zookeeper.ReadOnlyZKClient(139): Connect 0x63bbba5d to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:38,040 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5edf8a6d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:38,041 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:38,043 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39844, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:38,043 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x63bbba5d to 127.0.0.1:61565 2023-02-12 14:55:38,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:38,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(601): No existing snapshot, attempting snapshot... 2023-02-12 14:55:38,044 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(659): Table is disabled, running snapshot entirely on master for { ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-12 14:55:38,046 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-12 14:55:38,047 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=95, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-12 14:55:38,047 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=95, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE; timeout=600000, timestamp=1676214338047 2023-02-12 14:55:38,047 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(174): Running DISABLED table snapshot hbase-failsafe-snaptb-1676213723352-1676213738035 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink 2023-02-12 14:55:38,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(664): Started snapshot: { ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-12 14:55:38,048 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2418): Waiting a max of 300000 ms for snapshot '{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 }'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:38,048 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#1) Sleeping: 100ms while waiting for snapshot completion. 2023-02-12 14:55:38,143 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 14:55:38,148 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:38,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:38,149 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:38,149 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#2) Sleeping: 200ms while waiting for snapshot completion. 2023-02-12 14:55:38,349 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:38,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:38,350 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:38,351 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#3) Sleeping: 300ms while waiting for snapshot completion. 2023-02-12 14:55:38,453 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(97): Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=DISABLED ttl=0 } 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(297): Storing region-info for snapshot. 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-12 14:55:38,455 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(302): Creating references for hfiles 2023-02-12 14:55:38,458 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/b3f47004a60c47d7a21a470249efb1d0] hfiles 2023-02-12 14:55:38,458 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_ 2023-02-12 14:55:38,458 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/b3f47004a60c47d7a21a470249efb1d0 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/8efb885f124e47b29197be1ea0bf9cec] hfiles 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_ 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/c952dd9230b44b288a115e9b4b73f5b3] hfiles 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/2aa0989bffb34d159ed605901e7d0b39] hfiles 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/2): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_ 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/8efb885f124e47b29197be1ea0bf9cec 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (2/2): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/c952dd9230b44b288a115e9b4b73f5b3 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-0] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/2aa0989bffb34d159ed605901e7d0b39 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212e4fc573a4b9d4ceeb7d6887e3c490581, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302127078e27556344a0aa5c4c69746961093, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212a619aeb36ff447dc977e0f5ccec0a359] hfiles 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/3): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212e4fc573a4b9d4ceeb7d6887e3c490581 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/3): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302127078e27556344a0aa5c4c69746961093 2023-02-12 14:55:38,459 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/3): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212a619aeb36ff447dc977e0f5ccec0a359 2023-02-12 14:55:38,460 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/26b37568688f46b585aec1f729c5fcdd] hfiles 2023-02-12 14:55:38,460 DEBUG [DisabledTableSnapshot-pool-2] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/26b37568688f46b585aec1f729c5fcdd 2023-02-12 14:55:38,460 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/2e4db3ffeb224d03a93d1e88f994b790] hfiles 2023-02-12 14:55:38,460 DEBUG [DisabledTableSnapshot-pool-3] snapshot.SnapshotManifest(363): Adding reference for hfile (1/1): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/2e4db3ffeb224d03a93d1e88f994b790 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(355): Adding snapshot references for [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302123415c46c45d6470d9112000f8bf04c70, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302129b36e5cfbffa44e499012c03047dd37b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cc09253e7c7f4e1683b5d6f6775972fb, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cfe43cb34e8e4fb38507a4f4f3d42c4b_del, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212d4be49120391415bb2ea6fe55808e6b9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212283dbf899f7f49f1a537097648f293ad, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302125e1fe8592e164715b749d5d893383f7f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021277eedf1d464140ec88991fce862497b9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212af91ff60a0254fbfa2c8d9efcf4a313d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212e9e03c7faa9b493b8ba7a95e7b971683_del, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302120fb46ed2fc32453fbb3b6d6cec5237f6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302122cc3e41fa25843a1848eaeae94611883, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123748b931d9074f96b8c1f4dd73ecc85b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123f15fb6bd0f04990aba6745eee75a1c9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230212ae32d3fec30348d69aed4b465836169d_del] hfiles 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (1/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302123415c46c45d6470d9112000f8bf04c70 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (2/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b202302129b36e5cfbffa44e499012c03047dd37b 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (3/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cc09253e7c7f4e1683b5d6f6775972fb 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (4/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cfe43cb34e8e4fb38507a4f4f3d42c4b_del 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (5/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212d4be49120391415bb2ea6fe55808e6b9 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (6/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212283dbf899f7f49f1a537097648f293ad 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (7/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c202302125e1fe8592e164715b749d5d893383f7f 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (8/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c2023021277eedf1d464140ec88991fce862497b9 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (9/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212af91ff60a0254fbfa2c8d9efcf4a313d 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (10/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212e9e03c7faa9b493b8ba7a95e7b971683_del 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (11/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302120fb46ed2fc32453fbb3b6d6cec5237f6 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (12/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302122cc3e41fa25843a1848eaeae94611883 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (13/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123748b931d9074f96b8c1f4dd73ecc85b 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (14/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e202302123f15fb6bd0f04990aba6745eee75a1c9 2023-02-12 14:55:38,469 DEBUG [DisabledTableSnapshot-pool-1] snapshot.SnapshotManifest(363): Adding reference for hfile (15/15): hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230212ae32d3fec30348d69aed4b465836169d_del 2023-02-12 14:55:38,651 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:38,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:38,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:38,652 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#4) Sleeping: 500ms while waiting for snapshot completion. 2023-02-12 14:55:38,874 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.DisabledTableSnapshotHandler(120): Marking snapshot{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=DISABLED ttl=0 } as finished. 2023-02-12 14:55:38,874 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifest(484): Convert to Single Snapshot Manifest for hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:38,874 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotManifestV1(126): No regions under directory:hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:39,152 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:39,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:39,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(404): Snapshoting '{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' is still in progress! 2023-02-12 14:55:39,153 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2428): (#5) Sleeping: 1000ms while waiting for snapshot completion. 2023-02-12 14:55:39,301 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.SnapshotDescriptionUtils(404): Sentinel is done, just moving the snapshot from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676213723352-1676213738035 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:39,717 INFO [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(229): Snapshot hbase-failsafe-snaptb-1676213723352-1676213738035 of table testCompactionWithHFileLink completed 2023-02-12 14:55:39,717 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(246): Launching cleanup of working dir:hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:39,718 ERROR [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(251): Couldn't delete snapshot working directory:hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.hbase-snapshot/.tmp/hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:39,718 DEBUG [MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0-0] snapshot.TakeSnapshotHandler(257): Table snapshot journal : Running DISABLED table snapshot hbase-failsafe-snaptb-1676213723352-1676213738035 C_M_SNAPSHOT_TABLE on table testCompactionWithHFileLink at 1676213738047Starting to write region info and WALs for regions for offline snapshot:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=DISABLED ttl=0 } at 1676213738454 (+407 ms)Consolidate snapshot: hbase-failsafe-snaptb-1676213723352-1676213738035 at 1676213738874 (+420 ms)Loading Region manifests for hbase-failsafe-snaptb-1676213723352-1676213738035 at 1676213738874Writing data manifest for hbase-failsafe-snaptb-1676213723352-1676213738035 at 1676213738877 (+3 ms)Verifying snapshot: hbase-failsafe-snaptb-1676213723352-1676213738035 at 1676213739283 (+406 ms)Snapshot hbase-failsafe-snaptb-1676213723352-1676213738035 of table testCompactionWithHFileLink completed at 1676213739717 (+434 ms) 2023-02-12 14:55:39,719 DEBUG [PEWorker-5] locking.LockProcedure(242): UNLOCKED pid=95, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE 2023-02-12 14:55:39,720 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=95, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testCompactionWithHFileLink, type=EXCLUSIVE in 1.6730 sec 2023-02-12 14:55:40,154 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2434): Getting current status of snapshot from master... 2023-02-12 14:55:40,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1209): Checking to see if snapshot from request:{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 } is done 2023-02-12 14:55:40,155 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(401): Snapshot '{ ss=hbase-failsafe-snaptb-1676213723352-1676213738035 table=testCompactionWithHFileLink type=FLUSH ttl=0 }' has completed, notifying client. 2023-02-12 14:55:40,196 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:RESTORE_SNAPSHOT_PRE_OPERATION; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676213723352" table: "testCompactionWithHFileLink" creation_time: 1676213723362 type: FLUSH version: 2 ttl: 0 ) 2023-02-12 14:55:40,196 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(880): Restore snapshot=snaptb-1676213723352 as table=testCompactionWithHFileLink 2023-02-12 14:55:40,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-12 14:55:40,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-12 14:55:40,501 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-12 14:55:40,607 DEBUG [PEWorker-3] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-12 14:55:40,608 DEBUG [PEWorker-3] util.FSTableDescriptors(519): Deleted hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000001 2023-02-12 14:55:40,608 INFO [PEWorker-3] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/.tabledesc/.tableinfo.0000000002 2023-02-12 14:55:40,609 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(374): Starting restore snapshot={ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } 2023-02-12 14:55:40,613 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(177): starting restore table regions using snapshot=name: "snaptb-1676213723352" table: "testCompactionWithHFileLink" creation_time: 1676213723362 type: FLUSH version: 2 ttl: 0 2023-02-12 14:55:40,614 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(785): get table regions: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink 2023-02-12 14:55:40,618 DEBUG [PEWorker-3] snapshot.RestoreSnapshotHelper(797): found 3 regions for table=testCompactionWithHFileLink 2023-02-12 14:55:40,622 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302127078e27556344a0aa5c4c69746961093 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302127078e27556344a0aa5c4c69746961093 2023-02-12 14:55:40,623 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212e4fc573a4b9d4ceeb7d6887e3c490581 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212e4fc573a4b9d4ceeb7d6887e3c490581 2023-02-12 14:55:40,624 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212a619aeb36ff447dc977e0f5ccec0a359 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212a619aeb36ff447dc977e0f5ccec0a359 2023-02-12 14:55:40,624 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,627 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,630 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,632 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,635 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,637 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,640 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,643 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,646 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,649 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,652 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,654 DEBUG [RestoreSnapshot-pool-0] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 of snapshot=snaptb-1676213723352 to region=0e7b6fa9e81bd62588edd7d77d6165e4 table=testCompactionWithHFileLink 2023-02-12 14:55:40,659 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cfe43cb34e8e4fb38507a4f4f3d42c4b_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c4ca4238a0b923820dcc509a6f75849b20230212cfe43cb34e8e4fb38507a4f4f3d42c4b_del 2023-02-12 14:55:40,660 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212e9e03c7faa9b493b8ba7a95e7b971683_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/c81e728d9d4c2f636f067f89cc14862c20230212e9e03c7faa9b493b8ba7a95e7b971683_del 2023-02-12 14:55:40,661 DEBUG [RestoreSnapshot-pool-0] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230212ae32d3fec30348d69aed4b465836169d_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family2/d41d8cd98f00b204e9800998ecf8427e20230212ae32d3fec30348d69aed4b465836169d_del 2023-02-12 14:55:40,661 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:40,661 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:40,661 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(223): region to restore: c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:40,665 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/3d495b7e83f94227a87fb662ac5ed978_SeqId_29_ 2023-02-12 14:55:40,665 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/3cdeeb4c79e64fef88a21c317e8e388d_SeqId_29_ 2023-02-12 14:55:40,665 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/76b89c5898a04f56a30724d590f95297_SeqId_29_ 2023-02-12 14:55:40,667 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/b3f47004a60c47d7a21a470249efb1d0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/b3f47004a60c47d7a21a470249efb1d0 2023-02-12 14:55:40,667 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink e03728432e5947eda80697c61f891d10 of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,667 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/c952dd9230b44b288a115e9b4b73f5b3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/c952dd9230b44b288a115e9b4b73f5b3 2023-02-12 14:55:40,667 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink ecdc577f07da4cb1ad7ba1378a36e7d2 of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,667 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/8efb885f124e47b29197be1ea0bf9cec to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/8efb885f124e47b29197be1ea0bf9cec 2023-02-12 14:55:40,667 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 69f858874fa141889a3f3d14d75bc3fd of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,678 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 49707ccb4e4f44f28a5e46acdf4a01f4 of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,679 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 41be1e3dff234cadbea7808c2c672e0c of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,679 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5c12eba08d1240b9b5042b98c3072cf6 of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,683 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 526d1d45234641e5877c28f0ee1c4fe6 of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,683 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a0644edaa9354d88938f406a3dbc2943 of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,684 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 5ba1a1c8bb5046099d4c27b359ce7176 of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,688 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 89aadb319ba44c88a9214d1ad6836b68 of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,688 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 3e7296de83954248a5d807c9f40e18fa of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,689 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 6f3d54677c014906aa0b5cb62516a2c3 of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,696 DEBUG [RestoreSnapshot-pool-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/2aa0989bffb34d159ed605901e7d0b39 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/2aa0989bffb34d159ed605901e7d0b39 2023-02-12 14:55:40,696 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f2a416bc30b44a53874f33431ceee564 of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,696 DEBUG [RestoreSnapshot-pool-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/2e4db3ffeb224d03a93d1e88f994b790 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/2e4db3ffeb224d03a93d1e88f994b790 2023-02-12 14:55:40,696 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 18484b7a53624894893277beb20e6d58 of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,696 DEBUG [RestoreSnapshot-pool-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/26b37568688f46b585aec1f729c5fcdd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/26b37568688f46b585aec1f729c5fcdd 2023-02-12 14:55:40,696 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 39c69ce2cdc04e89a1b8fcc62e0e41b2 of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,699 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 20f9cf39b0594acf9ec77c570d62698c of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,699 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 700fda40ac8649ef9a44f4cfdc556a9f of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,700 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink a235bba083664fa998b43af265d0eed2 of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,702 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink f31845a05242422fb2c55797c8de37b4 of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,702 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 4aba417dd6d142129901d792ec2f7631 of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,703 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 30098feffd2e47808617c86382a508a5 of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,705 DEBUG [RestoreSnapshot-pool-3] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 66e244f61f03457384884137acf1b58b of snapshot=snaptb-1676213723352 to region=c330f294c6804231199503c93a0f53d8 table=testCompactionWithHFileLink 2023-02-12 14:55:40,705 DEBUG [RestoreSnapshot-pool-1] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 27b7d09543864486b8af9b45fb33b2f3 of snapshot=snaptb-1676213723352 to region=73ab03d290bf433ec02678876bd209ea table=testCompactionWithHFileLink 2023-02-12 14:55:40,706 DEBUG [RestoreSnapshot-pool-2] snapshot.RestoreSnapshotHelper(517): Restoring missing HFileLink 053d84dda82248ecaeb1d4d178f7710e of snapshot=snaptb-1676213723352 to region=7c8a55283e262bc196e8394e111484b6 table=testCompactionWithHFileLink 2023-02-12 14:55:40,710 INFO [PEWorker-3] snapshot.RestoreSnapshotHelper(266): finishing restore table regions using snapshot=name: "snaptb-1676213723352" table: "testCompactionWithHFileLink" creation_time: 1676213723362 type: FLUSH version: 2 ttl: 0 2023-02-12 14:55:40,711 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676213740711"}]},"ts":"9223372036854775807"} 2023-02-12 14:55:40,711 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676213740711"}]},"ts":"9223372036854775807"} 2023-02-12 14:55:40,711 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676213740711"}]},"ts":"9223372036854775807"} 2023-02-12 14:55:40,713 INFO [PEWorker-3] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-12 14:55:40,713 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 73ab03d290bf433ec02678876bd209ea, NAME => 'testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 7c8a55283e262bc196e8394e111484b6, NAME => 'testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => c330f294c6804231199503c93a0f53d8, NAME => 'testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.', STARTKEY => '', ENDKEY => '1'}] 2023-02-12 14:55:40,714 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213740712"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213740712"}]},"ts":"1676213740712"} 2023-02-12 14:55:40,714 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213740712"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213740712"}]},"ts":"1676213740712"} 2023-02-12 14:55:40,714 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213740712"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213740712"}]},"ts":"1676213740712"} 2023-02-12 14:55:40,715 INFO [PEWorker-3] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-12 14:55:40,715 INFO [PEWorker-3] hbase.MetaTableAccessor(1810): Overwritten 3 regions to Meta 2023-02-12 14:55:40,715 DEBUG [PEWorker-3] hbase.MetaTableAccessor(1811): Overwritten regions: [{ENCODED => 73ab03d290bf433ec02678876bd209ea, NAME => 'testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.', STARTKEY => '2', ENDKEY => ''}, {ENCODED => 7c8a55283e262bc196e8394e111484b6, NAME => 'testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.', STARTKEY => '1', ENDKEY => '2'}, {ENCODED => c330f294c6804231199503c93a0f53d8, NAME => 'testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.', STARTKEY => '', ENDKEY => '1'}] 2023-02-12 14:55:40,716 INFO [PEWorker-3] procedure.RestoreSnapshotProcedure(447): Restore snapshot={ ss=snaptb-1676213723352 table=testCompactionWithHFileLink type=FLUSH ttl=0 } on table=testCompactionWithHFileLink completed! 2023-02-12 14:55:40,717 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; RestoreSnapshotProcedure (table=testCompactionWithHFileLink snapshot=name: "snaptb-1676213723352" table: "testCompactionWithHFileLink" creation_time: 1676213723362 type: FLUSH version: 2 ttl: 0 ) in 529 msec 2023-02-12 14:55:40,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-12 14:55:40,802 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: MODIFY, Table Name: default:testCompactionWithHFileLink, procId: 96 completed 2023-02-12 14:55:40,802 INFO [Listener at localhost/37945] client.HBaseAdmin(2618): Deleting restore-failsafe snapshot: hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:40,806 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(748): Client=jenkins//172.31.14.131 delete name: "hbase-failsafe-snaptb-1676213723352-1676213738035" 2023-02-12 14:55:40,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] snapshot.SnapshotManager(333): Deleting snapshot: hbase-failsafe-snaptb-1676213723352-1676213738035 2023-02-12 14:55:40,811 INFO [Listener at localhost/37945] client.HBaseAdmin$14(830): Started enable of testCompactionWithHFileLink 2023-02-12 14:55:40,814 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.HMaster$10(2372): Client=jenkins//172.31.14.131 enable testCompactionWithHFileLink 2023-02-12 14:55:40,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=97, state=RUNNABLE:ENABLE_TABLE_PREPARE; EnableTableProcedure table=testCompactionWithHFileLink 2023-02-12 14:55:40,822 INFO [PEWorker-4] procedure.EnableTableProcedure(296): Attempting to enable the table testCompactionWithHFileLink 2023-02-12 14:55:40,822 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213740822"}]},"ts":"1676213740822"} 2023-02-12 14:55:40,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-12 14:55:40,823 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLING in hbase:meta 2023-02-12 14:55:40,826 DEBUG [PEWorker-4] procedure.EnableTableProcedure(107): No change in number of region replicas (configuredReplicaCount=1); assigning. 2023-02-12 14:55:40,826 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN}, {pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN}, {pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN}] 2023-02-12 14:55:40,827 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN 2023-02-12 14:55:40,827 INFO [PEWorker-2] procedure.MasterProcedureScheduler(727): Took xlock for pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN 2023-02-12 14:55:40,827 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN 2023-02-12 14:55:40,828 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=98, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-12 14:55:40,828 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=100, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-12 14:55:40,828 INFO [PEWorker-2] assignment.TransitRegionStateProcedure(193): Starting pid=99, ppid=97, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN; state=CLOSED, location=null; forceNewPlan=false, retain=false 2023-02-12 14:55:40,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-12 14:55:40,979 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=73ab03d290bf433ec02678876bd209ea, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:40,979 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=7c8a55283e262bc196e8394e111484b6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:40,979 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=c330f294c6804231199503c93a0f53d8, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:40,980 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213740979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213740979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213740979"}]},"ts":"1676213740979"} 2023-02-12 14:55:40,980 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213740979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213740979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213740979"}]},"ts":"1676213740979"} 2023-02-12 14:55:40,980 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213740979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213740979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213740979"}]},"ts":"1676213740979"} 2023-02-12 14:55:40,981 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=101, ppid=100, state=RUNNABLE; OpenRegionProcedure 73ab03d290bf433ec02678876bd209ea, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:40,982 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=102, ppid=98, state=RUNNABLE; OpenRegionProcedure c330f294c6804231199503c93a0f53d8, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:40,982 INFO [PEWorker-4] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=99, state=RUNNABLE; OpenRegionProcedure 7c8a55283e262bc196e8394e111484b6, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:41,124 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-12 14:55:41,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:41,135 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c330f294c6804231199503c93a0f53d8, NAME => 'testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.', STARTKEY => '', ENDKEY => '1'} 2023-02-12 14:55:41,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:41,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,136 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,137 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,138 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:41,138 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:41,138 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c330f294c6804231199503c93a0f53d8 columnFamilyName family1 2023-02-12 14:55:41,153 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-3e7296de83954248a5d807c9f40e18fa 2023-02-12 14:55:41,172 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-41be1e3dff234cadbea7808c2c672e0c 2023-02-12 14:55:41,177 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-526d1d45234641e5877c28f0ee1c4fe6 2023-02-12 14:55:41,182 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-69f858874fa141889a3f3d14d75bc3fd 2023-02-12 14:55:41,183 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(310): Store=c330f294c6804231199503c93a0f53d8/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:41,183 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,184 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2 2023-02-12 14:55:41,184 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2 2023-02-12 14:55:41,184 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c330f294c6804231199503c93a0f53d8 columnFamilyName family2 2023-02-12 14:55:41,192 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-18484b7a53624894893277beb20e6d58 2023-02-12 14:55:41,198 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-4aba417dd6d142129901d792ec2f7631 2023-02-12 14:55:41,204 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-66e244f61f03457384884137acf1b58b 2023-02-12 14:55:41,209 DEBUG [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-700fda40ac8649ef9a44f4cfdc556a9f 2023-02-12 14:55:41,210 INFO [StoreOpener-c330f294c6804231199503c93a0f53d8-1] regionserver.HStore(310): Store=c330f294c6804231199503c93a0f53d8/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:41,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:41,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c330f294c6804231199503c93a0f53d8 2023-02-12 14:55:41,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c330f294c6804231199503c93a0f53d8; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62885132, jitterRate=-0.06293851137161255}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:41,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c330f294c6804231199503c93a0f53d8: Running coprocessor pre-open hook at 1676213741136Writing region info on filesystem at 1676213741136Initializing all the Stores at 1676213741136Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213741137 (+1 ms)Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213741137Cleaning up temporary data from old regions at 1676213741212 (+75 ms)Cleaning up detritus from prior splits at 1676213741212Running coprocessor post-open hooks at 1676213741214 (+2 ms)Region opened successfully at 1676213741214 2023-02-12 14:55:41,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8., pid=102, masterSystemTime=1676213741133 2023-02-12 14:55:41,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:41,216 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 14:55:41,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 14:55:41,216 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 14:55:41,217 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:41,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:41,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:41,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7c8a55283e262bc196e8394e111484b6, NAME => 'testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.', STARTKEY => '1', ENDKEY => '2'} 2023-02-12 14:55:41,218 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=c330f294c6804231199503c93a0f53d8, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:41,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:41,218 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213741218"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213741218"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213741218"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213741218"}]},"ts":"1676213741218"} 2023-02-12 14:55:41,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,220 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-12 14:55:41,220 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,220 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1912): c330f294c6804231199503c93a0f53d8/family2 is initiating minor compaction (all files) 2023-02-12 14:55:41,220 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-12 14:55:41,220 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1912): c330f294c6804231199503c93a0f53d8/family1 is initiating minor compaction (all files) 2023-02-12 14:55:41,220 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegion(2258): Starting compaction of c330f294c6804231199503c93a0f53d8/family2 in testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:41,220 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of c330f294c6804231199503c93a0f53d8/family1 in testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:55:41,220 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-18484b7a53624894893277beb20e6d58, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-700fda40ac8649ef9a44f4cfdc556a9f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-4aba417dd6d142129901d792ec2f7631, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-66e244f61f03457384884137acf1b58b] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp, totalSize=20.4 K 2023-02-12 14:55:41,220 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-69f858874fa141889a3f3d14d75bc3fd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-41be1e3dff234cadbea7808c2c672e0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-526d1d45234641e5877c28f0ee1c4fe6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-3e7296de83954248a5d807c9f40e18fa] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp, totalSize=21.6 K 2023-02-12 14:55:41,221 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=102, resume processing ppid=98 2023-02-12 14:55:41,221 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:41,221 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:41,221 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=102, ppid=98, state=SUCCESS; OpenRegionProcedure c330f294c6804231199503c93a0f53d8, server=jenkins-hbase4.apache.org,45319,1676213617001 in 237 msec 2023-02-12 14:55:41,221 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-18484b7a53624894893277beb20e6d58, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716181 2023-02-12 14:55:41,221 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-69f858874fa141889a3f3d14d75bc3fd, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716181 2023-02-12 14:55:41,221 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c8a55283e262bc196e8394e111484b6 columnFamilyName family1 2023-02-12 14:55:41,222 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-700fda40ac8649ef9a44f4cfdc556a9f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717967 2023-02-12 14:55:41,222 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=c330f294c6804231199503c93a0f53d8, ASSIGN in 395 msec 2023-02-12 14:55:41,222 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-41be1e3dff234cadbea7808c2c672e0c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717967 2023-02-12 14:55:41,223 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-526d1d45234641e5877c28f0ee1c4fe6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719756 2023-02-12 14:55:41,223 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-4aba417dd6d142129901d792ec2f7631, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719756 2023-02-12 14:55:41,223 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-3e7296de83954248a5d807c9f40e18fa, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:41,225 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-66e244f61f03457384884137acf1b58b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:41,238 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5ba1a1c8bb5046099d4c27b359ce7176 2023-02-12 14:55:41,248 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5c12eba08d1240b9b5042b98c3072cf6 2023-02-12 14:55:41,261 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-6f3d54677c014906aa0b5cb62516a2c3 2023-02-12 14:55:41,269 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] throttle.PressureAwareThroughputController(145): c330f294c6804231199503c93a0f53d8#family2#compaction#113 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:41,270 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): c330f294c6804231199503c93a0f53d8#family1#compaction#114 average throughput is 0.29 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:41,279 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-ecdc577f07da4cb1ad7ba1378a36e7d2 2023-02-12 14:55:41,279 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(310): Store=7c8a55283e262bc196e8394e111484b6/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:41,280 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,281 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2 2023-02-12 14:55:41,281 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2 2023-02-12 14:55:41,281 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7c8a55283e262bc196e8394e111484b6 columnFamilyName family2 2023-02-12 14:55:41,291 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-053d84dda82248ecaeb1d4d178f7710e 2023-02-12 14:55:41,296 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-30098feffd2e47808617c86382a508a5 2023-02-12 14:55:41,305 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-39c69ce2cdc04e89a1b8fcc62e0e41b2 2023-02-12 14:55:41,311 DEBUG [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-a235bba083664fa998b43af265d0eed2 2023-02-12 14:55:41,311 INFO [StoreOpener-7c8a55283e262bc196e8394e111484b6-1] regionserver.HStore(310): Store=7c8a55283e262bc196e8394e111484b6/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:41,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:41,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:55:41,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 7c8a55283e262bc196e8394e111484b6; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66510014, jitterRate=-0.008923560380935669}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:41,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 7c8a55283e262bc196e8394e111484b6: Running coprocessor pre-open hook at 1676213741218Writing region info on filesystem at 1676213741218Initializing all the Stores at 1676213741219 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213741219Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213741220 (+1 ms)Cleaning up temporary data from old regions at 1676213741314 (+94 ms)Cleaning up detritus from prior splits at 1676213741314Running coprocessor post-open hooks at 1676213741317 (+3 ms)Region opened successfully at 1676213741317 2023-02-12 14:55:41,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6., pid=103, masterSystemTime=1676213741133 2023-02-12 14:55:41,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 14:55:41,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-12 14:55:41,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:41,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:41,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:41,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 73ab03d290bf433ec02678876bd209ea, NAME => 'testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.', STARTKEY => '2', ENDKEY => ''} 2023-02-12 14:55:41,320 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=99 updating hbase:meta row=7c8a55283e262bc196e8394e111484b6, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:41,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testCompactionWithHFileLink 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:41,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,320 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213741320"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213741320"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213741320"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213741320"}]},"ts":"1676213741320"} 2023-02-12 14:55:41,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,322 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,323 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:41,323 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:41,323 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73ab03d290bf433ec02678876bd209ea columnFamilyName family1 2023-02-12 14:55:41,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=99 2023-02-12 14:55:41,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=99, state=SUCCESS; OpenRegionProcedure 7c8a55283e262bc196e8394e111484b6, server=jenkins-hbase4.apache.org,45319,1676213617001 in 340 msec 2023-02-12 14:55:41,324 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=7c8a55283e262bc196e8394e111484b6, ASSIGN in 497 msec 2023-02-12 14:55:41,329 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-49707ccb4e4f44f28a5e46acdf4a01f4 2023-02-12 14:55:41,335 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-89aadb319ba44c88a9214d1ad6836b68 2023-02-12 14:55:41,340 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-a0644edaa9354d88938f406a3dbc2943 2023-02-12 14:55:41,345 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-e03728432e5947eda80697c61f891d10 2023-02-12 14:55:41,346 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(310): Store=73ab03d290bf433ec02678876bd209ea/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:41,346 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,347 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2 2023-02-12 14:55:41,347 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2 2023-02-12 14:55:41,347 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 73ab03d290bf433ec02678876bd209ea columnFamilyName family2 2023-02-12 14:55:41,354 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-20f9cf39b0594acf9ec77c570d62698c 2023-02-12 14:55:41,358 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-27b7d09543864486b8af9b45fb33b2f3 2023-02-12 14:55:41,363 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f2a416bc30b44a53874f33431ceee564 2023-02-12 14:55:41,369 DEBUG [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(539): loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f31845a05242422fb2c55797c8de37b4 2023-02-12 14:55:41,369 INFO [StoreOpener-73ab03d290bf433ec02678876bd209ea-1] regionserver.HStore(310): Store=73ab03d290bf433ec02678876bd209ea/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:41,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testCompactionWithHFileLink descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:41,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:55:41,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 73ab03d290bf433ec02678876bd209ea; next sequenceid=32; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62385238, jitterRate=-0.07038751244544983}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:41,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 73ab03d290bf433ec02678876bd209ea: Running coprocessor pre-open hook at 1676213741321Writing region info on filesystem at 1676213741321Initializing all the Stores at 1676213741321Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213741321Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213741321Cleaning up temporary data from old regions at 1676213741371 (+50 ms)Cleaning up detritus from prior splits at 1676213741371Running coprocessor post-open hooks at 1676213741373 (+2 ms)Region opened successfully at 1676213741373 2023-02-12 14:55:41,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea., pid=101, masterSystemTime=1676213741133 2023-02-12 14:55:41,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-12 14:55:41,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-12 14:55:41,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:41,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:41,376 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=100 updating hbase:meta row=73ab03d290bf433ec02678876bd209ea, regionState=OPEN, openSeqNum=32, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:41,376 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea.","families":{"info":[{"qualifier":"regioninfo","vlen":62,"tag":[],"timestamp":"1676213741376"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213741376"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213741376"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213741376"}]},"ts":"1676213741376"} 2023-02-12 14:55:41,378 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=101, resume processing ppid=100 2023-02-12 14:55:41,378 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=101, ppid=100, state=SUCCESS; OpenRegionProcedure 73ab03d290bf433ec02678876bd209ea, server=jenkins-hbase4.apache.org,45319,1676213617001 in 396 msec 2023-02-12 14:55:41,379 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-12 14:55:41,379 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; TransitRegionStateProcedure table=testCompactionWithHFileLink, region=73ab03d290bf433ec02678876bd209ea, ASSIGN in 552 msec 2023-02-12 14:55:41,380 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testCompactionWithHFileLink","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213741380"}]},"ts":"1676213741380"} 2023-02-12 14:55:41,381 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testCompactionWithHFileLink, state=ENABLED in hbase:meta 2023-02-12 14:55:41,382 INFO [PEWorker-4] procedure.EnableTableProcedure(310): Table 'testCompactionWithHFileLink' was successfully enabled. 2023-02-12 14:55:41,383 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=97, state=SUCCESS; EnableTableProcedure table=testCompactionWithHFileLink in 567 msec 2023-02-12 14:55:41,425 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=97 2023-02-12 14:55:41,426 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: ENABLE, Table Name: default:testCompactionWithHFileLink, procId: 97 completed 2023-02-12 14:55:41,437 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:41,445 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:41,455 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:41,456 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:41,463 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-12 14:55:41,478 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 12 selected files, and 0 irrelevant files table 'testCompactionWithHFileLink' and column 'family1' 2023-02-12 14:55:41,478 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:41,478 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(139): Connect 0x65f1e786 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:41,483 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@6a875512, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:41,483 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:41,483 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:41,483 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:42,093 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family1/bb4ef28cac0e4515bce0c925a4545e07 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/bb4ef28cac0e4515bce0c925a4545e07 2023-02-12 14:55:42,098 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testCompactionWithHFileLink' 2023-02-12 14:55:42,099 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c330f294c6804231199503c93a0f53d8/family1 of c330f294c6804231199503c93a0f53d8 into bb4ef28cac0e4515bce0c925a4545e07(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:42,099 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:42,099 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8., storeName=c330f294c6804231199503c93a0f53d8/family1, priority=12, startTime=1676213741215; duration=0sec 2023-02-12 14:55:42,099 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-12 14:55:42,100 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 14:55:42,102 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 22152 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-12 14:55:42,102 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1912): 7c8a55283e262bc196e8394e111484b6/family1 is initiating minor compaction (all files) 2023-02-12 14:55:42,102 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7c8a55283e262bc196e8394e111484b6/family1 in testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:42,102 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-ecdc577f07da4cb1ad7ba1378a36e7d2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5c12eba08d1240b9b5042b98c3072cf6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5ba1a1c8bb5046099d4c27b359ce7176, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-6f3d54677c014906aa0b5cb62516a2c3] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp, totalSize=21.6 K 2023-02-12 14:55:42,103 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-ecdc577f07da4cb1ad7ba1378a36e7d2, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716182 2023-02-12 14:55:42,103 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5c12eba08d1240b9b5042b98c3072cf6, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:42,104 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5ba1a1c8bb5046099d4c27b359ce7176, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:42,107 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-6f3d54677c014906aa0b5cb62516a2c3, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:42,130 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7c8a55283e262bc196e8394e111484b6#family1#compaction#115 average throughput is 1.17 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:42,170 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-69f858874fa141889a3f3d14d75bc3fd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-41be1e3dff234cadbea7808c2c672e0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-526d1d45234641e5877c28f0ee1c4fe6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-3e7296de83954248a5d807c9f40e18fa] to archive 2023-02-12 14:55:42,171 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:42,173 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-69f858874fa141889a3f3d14d75bc3fd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-69f858874fa141889a3f3d14d75bc3fd 2023-02-12 14:55:42,174 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-41be1e3dff234cadbea7808c2c672e0c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-41be1e3dff234cadbea7808c2c672e0c 2023-02-12 14:55:42,175 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-526d1d45234641e5877c28f0ee1c4fe6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-526d1d45234641e5877c28f0ee1c4fe6 2023-02-12 14:55:42,176 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-3e7296de83954248a5d807c9f40e18fa to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-3e7296de83954248a5d807c9f40e18fa 2023-02-12 14:55:42,372 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212acd4e8cf6b934c31b930ef1947cfb575 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/d41d8cd98f00b204e9800998ecf8427e20230212acd4e8cf6b934c31b930ef1947cfb575 2023-02-12 14:55:42,373 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302125b779bceedb144f4bcef2d8c7c9e4888 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c81e728d9d4c2f636f067f89cc14862c202302125b779bceedb144f4bcef2d8c7c9e4888 2023-02-12 14:55:42,373 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212636128624d224d619d14fcfa41699bce to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/c4ca4238a0b923820dcc509a6f75849b20230212636128624d224d619d14fcfa41699bce 2023-02-12 14:55:42,373 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:42,374 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:42,374 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:42,388 DEBUG [MobFileCompactionChore-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:42,392 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39852, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:42,400 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:55:42,408 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60720, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:55:42,422 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/34caf6164a904b7f864b8e248870173a first=Optional[0\x00\x00\x00\x00] last=Optional[0\x00\x00\x00\x07] 2023-02-12 14:55:42,422 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/3e41f172c6d64d5e9c98238634e5bf64 first=Optional[1\x00\x00\x00\x00] last=Optional[1\x00\x00\x00\x07] 2023-02-12 14:55:42,422 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/72198f8d78e44d52bb329a506fd5b8e5 first=Optional[2\x00\x00\x00\x00] last=Optional[2\x00\x00\x00\x07] 2023-02-12 14:55:42,423 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=32 for row 2 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/72198f8d78e44d52bb329a506fd5b8e5}] 2023-02-12 14:55:42,423 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=32 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/34caf6164a904b7f864b8e248870173a}] 2023-02-12 14:55:42,423 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=32 for row 1 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/3e41f172c6d64d5e9c98238634e5bf64}] 2023-02-12 14:55:42,428 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/3e41f172c6d64d5e9c98238634e5bf64 for inclusion in 7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:42,428 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/34caf6164a904b7f864b8e248870173a for inclusion in c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:42,428 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/72198f8d78e44d52bb329a506fd5b8e5 for inclusion in 73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:42,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x00 last=1\x00\x00\x00\x07 2023-02-12 14:55:42,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-12 14:55:42,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:42,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x00 last=0\x00\x00\x00\x07 2023-02-12 14:55:42,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x00 last=2\x00\x00\x00\x07 2023-02-12 14:55:42,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last=1 2023-02-12 14:55:42,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-12 14:55:42,431 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:42,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:42,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212/family1/3e41f172c6d64d5e9c98238634e5bf64 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__mbq7artlmjcc6o9922t54l788lo3m2mdttcl194n4ugqmm2g1u4elo94ibrs2kei/family1/3e41f172c6d64d5e9c98238634e5bf64 2023-02-12 14:55:42,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212/family1/72198f8d78e44d52bb329a506fd5b8e5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__18132rkutoe7uhf0o7k1jii3pjai2rufg17c8hibc525peou3dbvocvralddt78j/family1/72198f8d78e44d52bb329a506fd5b8e5 2023-02-12 14:55:42,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212/family1/34caf6164a904b7f864b8e248870173a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__l9n69cuhoe36l8loeu9fcqpo58lb2p1s0r0t84af3t8fqpukslmlji46kiidc3iq/family1/34caf6164a904b7f864b8e248870173a 2023-02-12 14:55:42,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__mbq7artlmjcc6o9922t54l788lo3m2mdttcl194n4ugqmm2g1u4elo94ibrs2kei/family1/3e41f172c6d64d5e9c98238634e5bf64 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63668bbdf5f6482b999e7c89489774a0_SeqId_34_ 2023-02-12 14:55:42,441 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__18132rkutoe7uhf0o7k1jii3pjai2rufg17c8hibc525peou3dbvocvralddt78j/family1/72198f8d78e44d52bb329a506fd5b8e5 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_ 2023-02-12 14:55:42,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__l9n69cuhoe36l8loeu9fcqpo58lb2p1s0r0t84af3t8fqpukslmlji46kiidc3iq/family1/34caf6164a904b7f864b8e248870173a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/167d986b64f64853948f61bb2edc3f67_SeqId_35_ 2023-02-12 14:55:42,442 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__mbq7artlmjcc6o9922t54l788lo3m2mdttcl194n4ugqmm2g1u4elo94ibrs2kei/family1/3e41f172c6d64d5e9c98238634e5bf64 into 7c8a55283e262bc196e8394e111484b6/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63668bbdf5f6482b999e7c89489774a0_SeqId_34_ - updating store file list. 2023-02-12 14:55:42,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__18132rkutoe7uhf0o7k1jii3pjai2rufg17c8hibc525peou3dbvocvralddt78j/family1/72198f8d78e44d52bb329a506fd5b8e5 into 73ab03d290bf433ec02678876bd209ea/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_ - updating store file list. 2023-02-12 14:55:42,443 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__l9n69cuhoe36l8loeu9fcqpo58lb2p1s0r0t84af3t8fqpukslmlji46kiidc3iq/family1/34caf6164a904b7f864b8e248870173a into c330f294c6804231199503c93a0f53d8/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/167d986b64f64853948f61bb2edc3f67_SeqId_35_ - updating store file list. 2023-02-12 14:55:42,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_ into 73ab03d290bf433ec02678876bd209ea/family1 2023-02-12 14:55:42,446 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__18132rkutoe7uhf0o7k1jii3pjai2rufg17c8hibc525peou3dbvocvralddt78j/family1/72198f8d78e44d52bb329a506fd5b8e5 into 73ab03d290bf433ec02678876bd209ea/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_) 2023-02-12 14:55:42,446 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63668bbdf5f6482b999e7c89489774a0_SeqId_34_ into 7c8a55283e262bc196e8394e111484b6/family1 2023-02-12 14:55:42,446 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__mbq7artlmjcc6o9922t54l788lo3m2mdttcl194n4ugqmm2g1u4elo94ibrs2kei/family1/3e41f172c6d64d5e9c98238634e5bf64 into 7c8a55283e262bc196e8394e111484b6/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/63668bbdf5f6482b999e7c89489774a0_SeqId_34_) 2023-02-12 14:55:42,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__18132rkutoe7uhf0o7k1jii3pjai2rufg17c8hibc525peou3dbvocvralddt78j/family1/72198f8d78e44d52bb329a506fd5b8e5 2023-02-12 14:55:42,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:42,447 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/167d986b64f64853948f61bb2edc3f67_SeqId_35_ into c330f294c6804231199503c93a0f53d8/family1 2023-02-12 14:55:42,447 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__l9n69cuhoe36l8loeu9fcqpo58lb2p1s0r0t84af3t8fqpukslmlji46kiidc3iq/family1/34caf6164a904b7f864b8e248870173a into c330f294c6804231199503c93a0f53d8/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family1/167d986b64f64853948f61bb2edc3f67_SeqId_35_) 2023-02-12 14:55:42,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__mbq7artlmjcc6o9922t54l788lo3m2mdttcl194n4ugqmm2g1u4elo94ibrs2kei/family1/3e41f172c6d64d5e9c98238634e5bf64 2023-02-12 14:55:42,448 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testCompactionWithHFileLink__l9n69cuhoe36l8loeu9fcqpo58lb2p1s0r0t84af3t8fqpukslmlji46kiidc3iq/family1/34caf6164a904b7f864b8e248870173a 2023-02-12 14:55:42,450 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:42,450 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:42,451 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:42,451 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:42,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 5 files of size 29606 starting at candidate #0 after considering 6 permutations with 6 in ratio 2023-02-12 14:55:42,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(1912): 73ab03d290bf433ec02678876bd209ea/family1 is initiating minor compaction (all files) 2023-02-12 14:55:42,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@1912f6a6; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-12 14:55:42,452 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b202302128418ad2e82a2483fa1932ff257a808d8 2023-02-12 14:55:42,453 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302124f4bfe99d019432f89f15f86eb8e7624 2023-02-12 14:55:42,454 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230212bd949186224e483e9e59ff1ad21b270b 2023-02-12 14:55:42,454 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:42,454 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302125a073fcdabb84585bedbb4d0a0e50785 2023-02-12 14:55:42,455 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:42,455 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230212d67802e157c741c185d97ee9d7932caf 2023-02-12 14:55:42,456 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e202302125d90297c13e44da883ca430350fb1ff1 2023-02-12 14:55:42,457 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c4ca4238a0b923820dcc509a6f75849b20230212ee6a081d2cd04e9b93d00c579b5534a8 2023-02-12 14:55:42,457 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c2023021245986ef10eaf450699d8e59eda6af068 2023-02-12 14:55:42,457 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-12 14:55:42,457 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c4ca4238a0b923820dcc509a6f75849b20230212' 2023-02-12 14:55:42,458 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:42,458 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-d41d8cd98f00b204e9800998ecf8427e20230212ff250aef8d944dcc814820b415648ac8 2023-02-12 14:55:42,458 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-12 14:55:42,458 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/d41d8cd98f00b204e9800998ecf8427e20230212' 2023-02-12 14:55:42,458 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c202302129a44703ccb2f48198da5fc939117bee3 2023-02-12 14:55:42,458 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:42,459 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230212d686d13faabd405d8e2689e6697ce6bc 2023-02-12 14:55:42,460 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/0e7b6fa9e81bd62588edd7d77d6165e4/family1/testCompactionWithHFileLink=0e7b6fa9e81bd62588edd7d77d6165e4-c81e728d9d4c2f636f067f89cc14862c20230212f6cbb248bc7543219f561bdc9af74dff 2023-02-12 14:55:42,460 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table testCompactionWithHFileLink 4 expired mob files are deleted 2023-02-12 14:55:42,460 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testCompactionWithHFileLink/c81e728d9d4c2f636f067f89cc14862c20230212' 2023-02-12 14:55:42,461 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230212 for table='testCompactionWithHFileLink' and column='family1' 2023-02-12 14:55:42,461 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testCompactionWithHFileLink' column='family1' 2023-02-12 14:55:42,465 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:42,468 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:42,471 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:42,472 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:42,478 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:42,479 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:42,495 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testCompactionWithHFileLink Thread=285 (was 236) - Thread LEAK? -, OpenFileDescriptor=761 (was 600) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=122 (was 166), ProcessCount=168 (was 168), AvailableMemoryMB=3701 (was 3835) 2023-02-12 14:55:42,500 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/.tmp/family2/94a9a61ccc3340b08b0093c9077f2aa6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/94a9a61ccc3340b08b0093c9077f2aa6 2023-02-12 14:55:42,505 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in c330f294c6804231199503c93a0f53d8/family2 of c330f294c6804231199503c93a0f53d8 into 94a9a61ccc3340b08b0093c9077f2aa6(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:42,505 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for c330f294c6804231199503c93a0f53d8: 2023-02-12 14:55:42,505 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8., storeName=c330f294c6804231199503c93a0f53d8/family2, priority=12, startTime=1676213741216; duration=1sec 2023-02-12 14:55:42,505 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=4), splitQueue=0 2023-02-12 14:55:42,505 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 73ab03d290bf433ec02678876bd209ea/family1 in testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:42,506 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-e03728432e5947eda80697c61f891d10, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-49707ccb4e4f44f28a5e46acdf4a01f4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-a0644edaa9354d88938f406a3dbc2943, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-89aadb319ba44c88a9214d1ad6836b68, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp, totalSize=28.9 K 2023-02-12 14:55:42,506 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-e03728432e5947eda80697c61f891d10, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716183 2023-02-12 14:55:42,507 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-49707ccb4e4f44f28a5e46acdf4a01f4, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:42,508 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-a0644edaa9354d88938f406a3dbc2943, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:42,508 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompaction Thread=284, OpenFileDescriptor=761, MaxFileDescriptor=60000, SystemLoadAverage=122, ProcessCount=168, AvailableMemoryMB=3700 2023-02-12 14:55:42,509 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-89aadb319ba44c88a9214d1ad6836b68, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:42,509 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_, keycount=16, bloomtype=ROW, size=7.3 K, encoding=NONE, compression=NONE, seqNum=35, earliestPutTs=1676213716183 2023-02-12 14:55:42,515 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.HMaster$15(2997): Client=jenkins//172.31.14.131 creating {NAME => 'ns'} 2023-02-12 14:55:42,516 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=104, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=ns 2023-02-12 14:55:42,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-12 14:55:42,523 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-12 14:55:42,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, state=SUCCESS; CreateNamespaceProcedure, namespace=ns in 10 msec 2023-02-12 14:55:42,528 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] throttle.PressureAwareThroughputController(145): 73ab03d290bf433ec02678876bd209ea#family1#compaction#116 average throughput is 0.78 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:42,570 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-18484b7a53624894893277beb20e6d58, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-700fda40ac8649ef9a44f4cfdc556a9f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-4aba417dd6d142129901d792ec2f7631, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-66e244f61f03457384884137acf1b58b] to archive 2023-02-12 14:55:42,571 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:42,572 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-18484b7a53624894893277beb20e6d58 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-18484b7a53624894893277beb20e6d58 2023-02-12 14:55:42,573 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-700fda40ac8649ef9a44f4cfdc556a9f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-700fda40ac8649ef9a44f4cfdc556a9f 2023-02-12 14:55:42,574 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-4aba417dd6d142129901d792ec2f7631 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-4aba417dd6d142129901d792ec2f7631 2023-02-12 14:55:42,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-66e244f61f03457384884137acf1b58b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/family2/testCompactionWithHFileLink=c330f294c6804231199503c93a0f53d8-66e244f61f03457384884137acf1b58b 2023-02-12 14:55:42,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=104 2023-02-12 14:55:42,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-12 14:55:42,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=ns:testMinorCompaction 2023-02-12 14:55:42,626 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:55:42,626 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "ns" qualifier: "testMinorCompaction" procId is: 105 2023-02-12 14:55:42,626 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:55:42,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 14:55:42,628 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:42,628 DEBUG [HFileArchiver-17] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:42,629 DEBUG [HFileArchiver-18] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:42,629 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c empty. 2023-02-12 14:55:42,629 DEBUG [HFileArchiver-18] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591 empty. 2023-02-12 14:55:42,629 DEBUG [HFileArchiver-17] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e empty. 2023-02-12 14:55:42,629 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:42,629 DEBUG [HFileArchiver-17] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:42,629 DEBUG [HFileArchiver-18] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:42,629 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived ns:testMinorCompaction regions 2023-02-12 14:55:42,728 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 14:55:42,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 14:55:42,946 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family1/4f5ca23da6ed4ff597db243a54ab1333 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/4f5ca23da6ed4ff597db243a54ab1333 2023-02-12 14:55:42,950 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7c8a55283e262bc196e8394e111484b6/family1 of 7c8a55283e262bc196e8394e111484b6 into 4f5ca23da6ed4ff597db243a54ab1333(size=7.6 K), total size for store is 14.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:42,950 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:42,950 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6., storeName=7c8a55283e262bc196e8394e111484b6/family1, priority=12, startTime=1676213741318; duration=0sec 2023-02-12 14:55:42,951 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=3), splitQueue=0 2023-02-12 14:55:42,951 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 14:55:42,953 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-12 14:55:42,953 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1912): 7c8a55283e262bc196e8394e111484b6/family2 is initiating minor compaction (all files) 2023-02-12 14:55:42,953 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 7c8a55283e262bc196e8394e111484b6/family2 in testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:55:42,953 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-39c69ce2cdc04e89a1b8fcc62e0e41b2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-a235bba083664fa998b43af265d0eed2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-30098feffd2e47808617c86382a508a5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-053d84dda82248ecaeb1d4d178f7710e] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp, totalSize=20.4 K 2023-02-12 14:55:42,953 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-39c69ce2cdc04e89a1b8fcc62e0e41b2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716182 2023-02-12 14:55:42,954 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-a235bba083664fa998b43af265d0eed2, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:42,954 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-30098feffd2e47808617c86382a508a5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:42,955 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-053d84dda82248ecaeb1d4d178f7710e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:42,967 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7c8a55283e262bc196e8394e111484b6#family2#compaction#117 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:42,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-ecdc577f07da4cb1ad7ba1378a36e7d2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5c12eba08d1240b9b5042b98c3072cf6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5ba1a1c8bb5046099d4c27b359ce7176, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-6f3d54677c014906aa0b5cb62516a2c3] to archive 2023-02-12 14:55:42,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:42,978 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-ecdc577f07da4cb1ad7ba1378a36e7d2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-ecdc577f07da4cb1ad7ba1378a36e7d2 2023-02-12 14:55:42,979 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5c12eba08d1240b9b5042b98c3072cf6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5c12eba08d1240b9b5042b98c3072cf6 2023-02-12 14:55:42,980 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5ba1a1c8bb5046099d4c27b359ce7176 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-5ba1a1c8bb5046099d4c27b359ce7176 2023-02-12 14:55:42,981 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-6f3d54677c014906aa0b5cb62516a2c3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family1/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-6f3d54677c014906aa0b5cb62516a2c3 2023-02-12 14:55:43,036 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/ns/testMinorCompaction/.tabledesc/.tableinfo.0000000001 2023-02-12 14:55:43,037 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(7671): creating {ENCODED => a9e7a8a10a47fb644927b2cbb23ae591, NAME => 'ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:55:43,038 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(7671): creating {ENCODED => 05e3c4702d173182c255943db0b5df6e, NAME => 'ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:55:43,038 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(7671): creating {ENCODED => c92bf76a80608abe270eff23b9822d0c, NAME => 'ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='ns:testMinorCompaction', {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:55:43,043 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:43,043 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1603): Closing a9e7a8a10a47fb644927b2cbb23ae591, disabling compactions & flushes 2023-02-12 14:55:43,043 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:43,043 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:43,043 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. after waiting 0 ms 2023-02-12 14:55:43,043 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:43,043 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:43,043 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-2] regionserver.HRegion(1557): Region close journal for a9e7a8a10a47fb644927b2cbb23ae591: Waiting for close lock at 1676213743043Disabling compacts and flushes for region at 1676213743043Disabling writes for close at 1676213743043Writing region close event to WAL at 1676213743043Closed at 1676213743043 2023-02-12 14:55:43,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 14:55:43,431 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:55:43,443 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:43,443 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1603): Closing c92bf76a80608abe270eff23b9822d0c, disabling compactions & flushes 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1603): Closing 05e3c4702d173182c255943db0b5df6e, disabling compactions & flushes 2023-02-12 14:55:43,444 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:43,444 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. after waiting 0 ms 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. after waiting 0 ms 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:43,444 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:43,444 INFO [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-1] regionserver.HRegion(1557): Region close journal for 05e3c4702d173182c255943db0b5df6e: Waiting for close lock at 1676213743444Disabling compacts and flushes for region at 1676213743444Disabling writes for close at 1676213743444Writing region close event to WAL at 1676213743444Closed at 1676213743444 2023-02-12 14:55:43,444 DEBUG [RegionOpenAndInit-ns:testMinorCompaction-pool-0] regionserver.HRegion(1557): Region close journal for c92bf76a80608abe270eff23b9822d0c: Waiting for close lock at 1676213743444Disabling compacts and flushes for region at 1676213743444Disabling writes for close at 1676213743444Writing region close event to WAL at 1676213743444Closed at 1676213743444 2023-02-12 14:55:43,446 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:55:43,447 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676213743447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213743447"}]},"ts":"1676213743447"} 2023-02-12 14:55:43,447 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676213743447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213743447"}]},"ts":"1676213743447"} 2023-02-12 14:55:43,447 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676213743447"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213743447"}]},"ts":"1676213743447"} 2023-02-12 14:55:43,449 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-12 14:55:43,450 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:55:43,450 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213743450"}]},"ts":"1676213743450"} 2023-02-12 14:55:43,451 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLING in hbase:meta 2023-02-12 14:55:43,454 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c92bf76a80608abe270eff23b9822d0c, ASSIGN}, {pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05e3c4702d173182c255943db0b5df6e, ASSIGN}, {pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9e7a8a10a47fb644927b2cbb23ae591, ASSIGN}] 2023-02-12 14:55:43,455 INFO [PEWorker-3] procedure.MasterProcedureScheduler(727): Took xlock for pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05e3c4702d173182c255943db0b5df6e, ASSIGN 2023-02-12 14:55:43,455 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c92bf76a80608abe270eff23b9822d0c, ASSIGN 2023-02-12 14:55:43,456 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9e7a8a10a47fb644927b2cbb23ae591, ASSIGN 2023-02-12 14:55:43,456 INFO [PEWorker-3] assignment.TransitRegionStateProcedure(193): Starting pid=107, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05e3c4702d173182c255943db0b5df6e, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:55:43,456 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c92bf76a80608abe270eff23b9822d0c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:55:43,456 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=108, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9e7a8a10a47fb644927b2cbb23ae591, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:55:43,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c92bf76a80608abe270eff23b9822d0c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:43,609 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=a9e7a8a10a47fb644927b2cbb23ae591, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:43,609 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=05e3c4702d173182c255943db0b5df6e, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:43,609 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676213743608"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213743608"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213743608"}]},"ts":"1676213743608"} 2023-02-12 14:55:43,609 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676213743608"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213743608"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213743608"}]},"ts":"1676213743608"} 2023-02-12 14:55:43,609 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676213743608"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213743608"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213743608"}]},"ts":"1676213743608"} 2023-02-12 14:55:43,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=109, ppid=106, state=RUNNABLE; OpenRegionProcedure c92bf76a80608abe270eff23b9822d0c, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:43,611 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=107, state=RUNNABLE; OpenRegionProcedure 05e3c4702d173182c255943db0b5df6e, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:43,611 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=108, state=RUNNABLE; OpenRegionProcedure a9e7a8a10a47fb644927b2cbb23ae591, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:55:43,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 14:55:43,747 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family1/5df675ca240d43c4a21bb68960828448 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/5df675ca240d43c4a21bb68960828448 2023-02-12 14:55:43,751 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1652): Completed compaction of 5 (all) file(s) in 73ab03d290bf433ec02678876bd209ea/family1 of 73ab03d290bf433ec02678876bd209ea into 5df675ca240d43c4a21bb68960828448(size=7.6 K), total size for store is 7.6 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:43,751 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:43,751 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea., storeName=family1, fileCount=5, fileSize=28.9 K (5.4 K, 5.4 K, 5.4 K, 5.4 K, 7.3 K), priority=2, time=1676213742451; duration=1sec 2023-02-12 14:55:43,751 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-12 14:55:43,752 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 14:55:43,754 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 20920 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-12 14:55:43,754 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1912): 73ab03d290bf433ec02678876bd209ea/family2 is initiating minor compaction (all files) 2023-02-12 14:55:43,754 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 73ab03d290bf433ec02678876bd209ea/family2 in testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:55:43,754 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f2a416bc30b44a53874f33431ceee564, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-20f9cf39b0594acf9ec77c570d62698c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f31845a05242422fb2c55797c8de37b4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-27b7d09543864486b8af9b45fb33b2f3] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp, totalSize=20.4 K 2023-02-12 14:55:43,755 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f2a416bc30b44a53874f33431ceee564, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213716183 2023-02-12 14:55:43,755 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-20f9cf39b0594acf9ec77c570d62698c, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213717968 2023-02-12 14:55:43,756 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f31845a05242422fb2c55797c8de37b4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213719757 2023-02-12 14:55:43,756 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] compactions.Compactor(207): Compacting testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-27b7d09543864486b8af9b45fb33b2f3, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213721543 2023-02-12 14:55:43,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:43,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 05e3c4702d173182c255943db0b5df6e, NAME => 'ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e.', STARTKEY => '1', ENDKEY => '2'} 2023-02-12 14:55:43,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction 05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:43,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,766 INFO [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,767 DEBUG [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1 2023-02-12 14:55:43,767 DEBUG [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1 2023-02-12 14:55:43,767 INFO [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 05e3c4702d173182c255943db0b5df6e columnFamilyName family1 2023-02-12 14:55:43,769 INFO [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] regionserver.HStore(310): Store=05e3c4702d173182c255943db0b5df6e/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:43,769 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] throttle.PressureAwareThroughputController(145): 73ab03d290bf433ec02678876bd209ea#family2#compaction#118 average throughput is 0.58 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:43,769 INFO [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,770 DEBUG [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2 2023-02-12 14:55:43,770 DEBUG [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2 2023-02-12 14:55:43,770 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-e03728432e5947eda80697c61f891d10, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-49707ccb4e4f44f28a5e46acdf4a01f4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-a0644edaa9354d88938f406a3dbc2943, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-89aadb319ba44c88a9214d1ad6836b68, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_] to archive 2023-02-12 14:55:43,770 INFO [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 05e3c4702d173182c255943db0b5df6e columnFamilyName family2 2023-02-12 14:55:43,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:43,772 INFO [StoreOpener-05e3c4702d173182c255943db0b5df6e-1] regionserver.HStore(310): Store=05e3c4702d173182c255943db0b5df6e/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:43,772 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-e03728432e5947eda80697c61f891d10 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-e03728432e5947eda80697c61f891d10 2023-02-12 14:55:43,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,773 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-49707ccb4e4f44f28a5e46acdf4a01f4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-49707ccb4e4f44f28a5e46acdf4a01f4 2023-02-12 14:55:43,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:43,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-a0644edaa9354d88938f406a3dbc2943 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-a0644edaa9354d88938f406a3dbc2943 2023-02-12 14:55:43,776 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 05e3c4702d173182c255943db0b5df6e 2023-02-12 14:55:43,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-89aadb319ba44c88a9214d1ad6836b68 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-89aadb319ba44c88a9214d1ad6836b68 2023-02-12 14:55:43,778 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family1/ec8940a5f45d4aafb8b8f9f4fccd3d65_SeqId_34_ 2023-02-12 14:55:43,778 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:55:43,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 05e3c4702d173182c255943db0b5df6e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64170726, jitterRate=-0.043781667947769165}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:43,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 05e3c4702d173182c255943db0b5df6e: Running coprocessor pre-open hook at 1676213743765Writing region info on filesystem at 1676213743765Initializing all the Stores at 1676213743766 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213743766Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213743766Cleaning up temporary data from old regions at 1676213743774 (+8 ms)Cleaning up detritus from prior splits at 1676213743774Running coprocessor post-open hooks at 1676213743779 (+5 ms)Region opened successfully at 1676213743779 2023-02-12 14:55:43,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e., pid=110, masterSystemTime=1676213743762 2023-02-12 14:55:43,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:43,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:43,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:43,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => c92bf76a80608abe270eff23b9822d0c, NAME => 'ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c.', STARTKEY => '', ENDKEY => '1'} 2023-02-12 14:55:43,781 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=107 updating hbase:meta row=05e3c4702d173182c255943db0b5df6e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:43,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,781 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e.","families":{"info":[{"qualifier":"regioninfo","vlen":50,"tag":[],"timestamp":"1676213743781"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213743781"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213743781"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213743781"}]},"ts":"1676213743781"} 2023-02-12 14:55:43,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:43,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,782 INFO [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,782 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/.tmp/family2/6d14b677ad654d0c834c2e5fef8cb3b2 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/6d14b677ad654d0c834c2e5fef8cb3b2 2023-02-12 14:55:43,784 DEBUG [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1 2023-02-12 14:55:43,784 DEBUG [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1 2023-02-12 14:55:43,784 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=107 2023-02-12 14:55:43,784 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=107, state=SUCCESS; OpenRegionProcedure 05e3c4702d173182c255943db0b5df6e, server=jenkins-hbase4.apache.org,45319,1676213617001 in 171 msec 2023-02-12 14:55:43,784 INFO [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c92bf76a80608abe270eff23b9822d0c columnFamilyName family1 2023-02-12 14:55:43,785 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=05e3c4702d173182c255943db0b5df6e, ASSIGN in 330 msec 2023-02-12 14:55:43,785 INFO [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] regionserver.HStore(310): Store=c92bf76a80608abe270eff23b9822d0c/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:43,785 INFO [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,786 DEBUG [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2 2023-02-12 14:55:43,786 DEBUG [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2 2023-02-12 14:55:43,787 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7c8a55283e262bc196e8394e111484b6/family2 of 7c8a55283e262bc196e8394e111484b6 into 6d14b677ad654d0c834c2e5fef8cb3b2(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:43,787 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 7c8a55283e262bc196e8394e111484b6: 2023-02-12 14:55:43,787 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6., storeName=7c8a55283e262bc196e8394e111484b6/family2, priority=12, startTime=1676213741318; duration=0sec 2023-02-12 14:55:43,787 INFO [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c92bf76a80608abe270eff23b9822d0c columnFamilyName family2 2023-02-12 14:55:43,787 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-12 14:55:43,787 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-12 14:55:43,787 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-02-12 14:55:43,787 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-02-12 14:55:43,787 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit(415): Not compacting testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. because compaction request was cancelled 2023-02-12 14:55:43,788 INFO [StoreOpener-c92bf76a80608abe270eff23b9822d0c-1] regionserver.HStore(310): Store=c92bf76a80608abe270eff23b9822d0c/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:43,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,789 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:43,791 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:55:43,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:55:43,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened c92bf76a80608abe270eff23b9822d0c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71068180, jitterRate=0.05899840593338013}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:43,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for c92bf76a80608abe270eff23b9822d0c: Running coprocessor pre-open hook at 1676213743782Writing region info on filesystem at 1676213743782Initializing all the Stores at 1676213743782Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213743782Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213743782Cleaning up temporary data from old regions at 1676213743789 (+7 ms)Cleaning up detritus from prior splits at 1676213743790 (+1 ms)Running coprocessor post-open hooks at 1676213743793 (+3 ms)Region opened successfully at 1676213743793 2023-02-12 14:55:43,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c., pid=109, masterSystemTime=1676213743762 2023-02-12 14:55:43,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:43,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:43,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:43,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a9e7a8a10a47fb644927b2cbb23ae591, NAME => 'ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591.', STARTKEY => '2', ENDKEY => ''} 2023-02-12 14:55:43,795 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=c92bf76a80608abe270eff23b9822d0c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:43,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompaction a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:55:43,795 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676213743795"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213743795"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213743795"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213743795"}]},"ts":"1676213743795"} 2023-02-12 14:55:43,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,796 INFO [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,797 DEBUG [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1 2023-02-12 14:55:43,797 DEBUG [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1 2023-02-12 14:55:43,797 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=109, resume processing ppid=106 2023-02-12 14:55:43,797 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=109, ppid=106, state=SUCCESS; OpenRegionProcedure c92bf76a80608abe270eff23b9822d0c, server=jenkins-hbase4.apache.org,45319,1676213617001 in 186 msec 2023-02-12 14:55:43,797 INFO [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9e7a8a10a47fb644927b2cbb23ae591 columnFamilyName family1 2023-02-12 14:55:43,798 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=c92bf76a80608abe270eff23b9822d0c, ASSIGN in 343 msec 2023-02-12 14:55:43,798 INFO [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] regionserver.HStore(310): Store=a9e7a8a10a47fb644927b2cbb23ae591/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:43,798 INFO [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,799 DEBUG [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2 2023-02-12 14:55:43,799 DEBUG [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2 2023-02-12 14:55:43,800 INFO [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9e7a8a10a47fb644927b2cbb23ae591 columnFamilyName family2 2023-02-12 14:55:43,800 INFO [StoreOpener-a9e7a8a10a47fb644927b2cbb23ae591-1] regionserver.HStore(310): Store=a9e7a8a10a47fb644927b2cbb23ae591/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:55:43,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table ns:testMinorCompaction descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:55:43,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:55:43,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:55:43,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a9e7a8a10a47fb644927b2cbb23ae591; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70266939, jitterRate=0.04705898463726044}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:55:43,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a9e7a8a10a47fb644927b2cbb23ae591: Running coprocessor pre-open hook at 1676213743795Writing region info on filesystem at 1676213743795Initializing all the Stores at 1676213743796 (+1 ms)Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213743796Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '5', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213743796Cleaning up temporary data from old regions at 1676213743801 (+5 ms)Cleaning up detritus from prior splits at 1676213743802 (+1 ms)Running coprocessor post-open hooks at 1676213743805 (+3 ms)Region opened successfully at 1676213743805 2023-02-12 14:55:43,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591., pid=111, masterSystemTime=1676213743762 2023-02-12 14:55:43,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:43,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:43,806 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=108 updating hbase:meta row=a9e7a8a10a47fb644927b2cbb23ae591, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:43,806 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591.","families":{"info":[{"qualifier":"regioninfo","vlen":49,"tag":[],"timestamp":"1676213743806"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213743806"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213743806"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213743806"}]},"ts":"1676213743806"} 2023-02-12 14:55:43,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=108 2023-02-12 14:55:43,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=108, state=SUCCESS; OpenRegionProcedure a9e7a8a10a47fb644927b2cbb23ae591, server=jenkins-hbase4.apache.org,45319,1676213617001 in 196 msec 2023-02-12 14:55:43,810 INFO [PEWorker-3] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=108, resume processing ppid=105 2023-02-12 14:55:43,810 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=108, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=ns:testMinorCompaction, region=a9e7a8a10a47fb644927b2cbb23ae591, ASSIGN in 355 msec 2023-02-12 14:55:43,811 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:55:43,811 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"ns:testMinorCompaction","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213743811"}]},"ts":"1676213743811"} 2023-02-12 14:55:43,812 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=ns:testMinorCompaction, state=ENABLED in hbase:meta 2023-02-12 14:55:43,814 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=105, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=ns:testMinorCompaction execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:55:43,815 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; CreateTableProcedure table=ns:testMinorCompaction in 1.1900 sec 2023-02-12 14:55:43,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-39c69ce2cdc04e89a1b8fcc62e0e41b2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-a235bba083664fa998b43af265d0eed2, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-30098feffd2e47808617c86382a508a5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-053d84dda82248ecaeb1d4d178f7710e] to archive 2023-02-12 14:55:43,870 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:43,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-39c69ce2cdc04e89a1b8fcc62e0e41b2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-39c69ce2cdc04e89a1b8fcc62e0e41b2 2023-02-12 14:55:43,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-a235bba083664fa998b43af265d0eed2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-a235bba083664fa998b43af265d0eed2 2023-02-12 14:55:43,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-30098feffd2e47808617c86382a508a5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-30098feffd2e47808617c86382a508a5 2023-02-12 14:55:43,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-053d84dda82248ecaeb1d4d178f7710e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/family2/testCompactionWithHFileLink=7c8a55283e262bc196e8394e111484b6-053d84dda82248ecaeb1d4d178f7710e 2023-02-12 14:55:44,133 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'ns:testMinorCompaction' 2023-02-12 14:55:44,733 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-12 14:55:44,733 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: ns:testMinorCompaction, procId: 105 completed 2023-02-12 14:55:44,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:44,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:44,740 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:44,742 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:44,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-12 14:55:44,744 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-12 14:55:44,744 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:44,744 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-12 14:55:44,744 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-12 14:55:44,744 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:44,744 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:44,746 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:44,746 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:44,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:44,747 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:44,747 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:44,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:44,747 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-12 14:55:44,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-12 14:55:44,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-12 14:55:44,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:44,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-12 14:55:44,748 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:44,748 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. started... 2023-02-12 14:55:44,748 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:44,748 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:44,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:44,750 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:44,750 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e7a8a10a47fb644927b2cbb23ae591 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:55:44,750 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. started... 2023-02-12 14:55:44,748 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. started... 2023-02-12 14:55:44,750 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c92bf76a80608abe270eff23b9822d0c 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:55:44,750 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 05e3c4702d173182c255943db0b5df6e 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:55:44,988 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/.tmp/family2/b647abc2cd884c398e900ce781651e20 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/b647abc2cd884c398e900ce781651e20 2023-02-12 14:55:44,993 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 73ab03d290bf433ec02678876bd209ea/family2 of 73ab03d290bf433ec02678876bd209ea into b647abc2cd884c398e900ce781651e20(size=6.4 K), total size for store is 6.4 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:44,993 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 73ab03d290bf433ec02678876bd209ea: 2023-02-12 14:55:44,993 INFO [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea., storeName=73ab03d290bf433ec02678876bd209ea/family2, priority=12, startTime=1676213741374; duration=1sec 2023-02-12 14:55:44,993 DEBUG [RS:0;jenkins-hbase4:45319-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:55:45,070 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f2a416bc30b44a53874f33431ceee564, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-20f9cf39b0594acf9ec77c570d62698c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f31845a05242422fb2c55797c8de37b4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-27b7d09543864486b8af9b45fb33b2f3] to archive 2023-02-12 14:55:45,071 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:45,072 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f2a416bc30b44a53874f33431ceee564 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f2a416bc30b44a53874f33431ceee564 2023-02-12 14:55:45,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-20f9cf39b0594acf9ec77c570d62698c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-20f9cf39b0594acf9ec77c570d62698c 2023-02-12 14:55:45,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f31845a05242422fb2c55797c8de37b4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-f31845a05242422fb2c55797c8de37b4 2023-02-12 14:55:45,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-27b7d09543864486b8af9b45fb33b2f3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/family2/testCompactionWithHFileLink=73ab03d290bf433ec02678876bd209ea-27b7d09543864486b8af9b45fb33b2f3 2023-02-12 14:55:45,172 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212e145c87a0e4f48f9a49a8dd316808056 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230212e145c87a0e4f48f9a49a8dd316808056 2023-02-12 14:55:45,174 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212fe1e5590b3764abd856ab75065de672a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230212fe1e5590b3764abd856ab75065de672a 2023-02-12 14:55:45,176 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021222d5c3e9b9c246f388ee42e7e371681b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021222d5c3e9b9c246f388ee42e7e371681b 2023-02-12 14:55:45,579 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/bfc399a98594442780869197a4dd007c 2023-02-12 14:55:45,586 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/7f28702439df4e23b24224f0556a2d93 2023-02-12 14:55:45,588 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/928d9bed69fb4d249b5091aeaa1234e0 2023-02-12 14:55:45,993 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302123dceb29d95314023a7751f07f51d1db1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302123dceb29d95314023a7751f07f51d1db1 2023-02-12 14:55:45,999 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302121466fbf414d1453a8acfa228db64d4a6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c202302121466fbf414d1453a8acfa228db64d4a6 2023-02-12 14:55:46,001 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021264a6b4295d2c42f6ac17ad27f3cee56d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b2023021264a6b4295d2c42f6ac17ad27f3cee56d 2023-02-12 14:55:46,398 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/f893f2d847e0428cb10bfcbfb20365b6 2023-02-12 14:55:46,402 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/bfc399a98594442780869197a4dd007c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/bfc399a98594442780869197a4dd007c 2023-02-12 14:55:46,403 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/aa195761aca84c9d89c1b86615b0b1f9 2023-02-12 14:55:46,405 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/bfc399a98594442780869197a4dd007c, entries=4, sequenceid=5, filesize=5.4 K 2023-02-12 14:55:46,406 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/f893f2d847e0428cb10bfcbfb20365b6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/f893f2d847e0428cb10bfcbfb20365b6 2023-02-12 14:55:46,408 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/7f28702439df4e23b24224f0556a2d93 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/7f28702439df4e23b24224f0556a2d93 2023-02-12 14:55:46,410 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/f893f2d847e0428cb10bfcbfb20365b6, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:55:46,411 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for c92bf76a80608abe270eff23b9822d0c in 1661ms, sequenceid=5, compaction requested=false 2023-02-12 14:55:46,412 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:46,412 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:46,412 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/7f28702439df4e23b24224f0556a2d93, entries=4, sequenceid=5, filesize=5.4 K 2023-02-12 14:55:46,412 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:46,412 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/2d78fed16b004973b6581a70b2280257 2023-02-12 14:55:46,412 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/aa195761aca84c9d89c1b86615b0b1f9 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/aa195761aca84c9d89c1b86615b0b1f9 2023-02-12 14:55:46,416 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/928d9bed69fb4d249b5091aeaa1234e0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/928d9bed69fb4d249b5091aeaa1234e0 2023-02-12 14:55:46,416 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/aa195761aca84c9d89c1b86615b0b1f9, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:55:46,416 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a9e7a8a10a47fb644927b2cbb23ae591 in 1666ms, sequenceid=5, compaction requested=false 2023-02-12 14:55:46,416 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:46,416 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:46,416 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:46,419 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/928d9bed69fb4d249b5091aeaa1234e0, entries=4, sequenceid=5, filesize=5.4 K 2023-02-12 14:55:46,419 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/2d78fed16b004973b6581a70b2280257 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/2d78fed16b004973b6581a70b2280257 2023-02-12 14:55:46,422 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/2d78fed16b004973b6581a70b2280257, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:55:46,423 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 05e3c4702d173182c255943db0b5df6e in 1673ms, sequenceid=5, compaction requested=false 2023-02-12 14:55:46,423 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:46,423 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:46,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:46,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:46,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-12 14:55:46,423 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-12 14:55:46,425 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:46,425 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:46,425 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,425 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:46,427 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-12 14:55:46,427 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38870a3f[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:46,427 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-12 14:55:46,427 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,428 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,428 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-12 14:55:46,428 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-12 14:55:46,428 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:46,428 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,428 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-12 14:55:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:46,429 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,429 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-12 14:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:46,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:46,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:46,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:46,432 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-12 14:55:46,432 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-12 14:55:46,432 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:46,432 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:46,432 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-12 14:55:46,432 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:46,434 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,434 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,434 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,434 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:46,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:46,435 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:46,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:46,436 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:46,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:46,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:46,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:46,438 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-12 14:55:46,438 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:46,438 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:46,438 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:46,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:46,438 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-12 14:55:46,538 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:46,539 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:46,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:46,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:46,541 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:46,543 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:46,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:46,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-12 14:55:46,545 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-12 14:55:46,545 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:46,545 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-12 14:55:46,545 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-12 14:55:46,545 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,545 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:46,546 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,546 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:46,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:46,546 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:46,546 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:46,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:46,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:46,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:46,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:46,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-12 14:55:46,547 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-12 14:55:46,547 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-12 14:55:46,547 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-12 14:55:46,547 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:46,547 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-12 14:55:46,547 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:46,547 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. started... 2023-02-12 14:55:46,548 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:46,548 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:46,548 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:46,548 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. started... 2023-02-12 14:55:46,548 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:46,548 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e7a8a10a47fb644927b2cbb23ae591 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:55:46,548 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. started... 2023-02-12 14:55:46,548 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c92bf76a80608abe270eff23b9822d0c 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:55:46,548 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 05e3c4702d173182c255943db0b5df6e 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:55:46,960 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212d80fa0b7703a470e977e56b8c6ef93f2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230212d80fa0b7703a470e977e56b8c6ef93f2 2023-02-12 14:55:46,964 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021231943ab2d0764b5da5eb2ddd5f2ce374 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021231943ab2d0764b5da5eb2ddd5f2ce374 2023-02-12 14:55:46,964 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212cf8357185dde4fbf8516b9f8f83e7e85 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230212cf8357185dde4fbf8516b9f8f83e7e85 2023-02-12 14:55:47,366 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/244da5eeee364f48921ba11bf01995b5 2023-02-12 14:55:47,369 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/4bbbc69d6c7c4419980c3723d7bd92fe 2023-02-12 14:55:47,369 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/2d50c2bae4214877b9a3010fbd9a2841 2023-02-12 14:55:47,784 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302129945795e66ef4628bdf73045e6316103 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302129945795e66ef4628bdf73045e6316103 2023-02-12 14:55:47,788 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302123c2f69ac199e4af89c610afbbf9429a1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b202302123c2f69ac199e4af89c610afbbf9429a1 2023-02-12 14:55:47,793 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212e4f72327df6845aa9b444e31e1f5348b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230212e4f72327df6845aa9b444e31e1f5348b 2023-02-12 14:55:47,798 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9 2023-02-12 14:55:47,801 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/244da5eeee364f48921ba11bf01995b5 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/244da5eeee364f48921ba11bf01995b5 2023-02-12 14:55:47,803 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/244da5eeee364f48921ba11bf01995b5, entries=4, sequenceid=9, filesize=5.4 K 2023-02-12 14:55:47,804 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9 2023-02-12 14:55:47,807 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:55:47,807 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for c92bf76a80608abe270eff23b9822d0c in 1259ms, sequenceid=9, compaction requested=false 2023-02-12 14:55:47,807 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:47,807 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:47,807 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:48,193 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/81ce6782b8f74be8909850afec94668b 2023-02-12 14:55:48,197 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/2d50c2bae4214877b9a3010fbd9a2841 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/2d50c2bae4214877b9a3010fbd9a2841 2023-02-12 14:55:48,198 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91 2023-02-12 14:55:48,201 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/2d50c2bae4214877b9a3010fbd9a2841, entries=4, sequenceid=9, filesize=5.4 K 2023-02-12 14:55:48,201 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/4bbbc69d6c7c4419980c3723d7bd92fe as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/4bbbc69d6c7c4419980c3723d7bd92fe 2023-02-12 14:55:48,202 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/81ce6782b8f74be8909850afec94668b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/81ce6782b8f74be8909850afec94668b 2023-02-12 14:55:48,205 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/4bbbc69d6c7c4419980c3723d7bd92fe, entries=4, sequenceid=9, filesize=5.4 K 2023-02-12 14:55:48,206 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/81ce6782b8f74be8909850afec94668b, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:55:48,206 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91 2023-02-12 14:55:48,206 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 05e3c4702d173182c255943db0b5df6e in 1658ms, sequenceid=9, compaction requested=false 2023-02-12 14:55:48,207 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:48,207 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:48,207 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:48,210 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:55:48,210 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a9e7a8a10a47fb644927b2cbb23ae591 in 1662ms, sequenceid=9, compaction requested=false 2023-02-12 14:55:48,210 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:48,210 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:48,210 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:48,210 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:48,211 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,211 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-12 14:55:48,211 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-12 14:55:48,212 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,212 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:48,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:48,212 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,212 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-12 14:55:48,214 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47faaf06[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:48,214 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-12 14:55:48,214 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,215 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,215 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,215 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,215 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-12 14:55:48,215 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-12 14:55:48,215 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,215 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:48,215 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:48,220 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,220 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-12 14:55:48,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:48,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:48,220 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:48,220 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-12 14:55:48,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:48,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:48,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:48,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-12 14:55:48,222 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-12 14:55:48,222 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:48,222 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:48,222 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-12 14:55:48,222 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:48,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:48,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,224 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:48,224 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:48,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:48,225 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:48,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:48,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,227 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:48,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:48,228 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-12 14:55:48,228 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:48,228 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-12 14:55:48,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:48,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:48,228 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,328 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:48,329 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:48,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:48,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:48,330 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:48,332 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:48,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:48,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-12 14:55:48,333 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-12 14:55:48,334 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:48,334 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-12 14:55:48,334 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-12 14:55:48,334 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,334 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:48,336 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:48,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:48,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:48,336 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:48,336 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:48,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:48,337 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:48,337 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-12 14:55:48,337 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-12 14:55:48,337 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-12 14:55:48,337 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-12 14:55:48,337 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:48,337 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-12 14:55:48,338 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:48,338 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. started... 2023-02-12 14:55:48,338 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:48,338 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:48,338 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e7a8a10a47fb644927b2cbb23ae591 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:55:48,338 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. started... 2023-02-12 14:55:48,338 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:48,338 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. started... 2023-02-12 14:55:48,338 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 05e3c4702d173182c255943db0b5df6e 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:55:48,338 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c92bf76a80608abe270eff23b9822d0c 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:55:48,757 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212bb7389f55e6a444fb0414405861522b2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230212bb7389f55e6a444fb0414405861522b2 2023-02-12 14:55:48,758 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212ba9b7af38fe14b2f91a34b5f0e896016 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230212ba9b7af38fe14b2f91a34b5f0e896016 2023-02-12 14:55:48,758 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021215626d7c06dd43ea8ebc9580a64b4b49 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e2023021215626d7c06dd43ea8ebc9580a64b4b49 2023-02-12 14:55:48,763 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/b130d1bae975465db7a9a1e3b7f80591 2023-02-12 14:55:49,164 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/659f32103acf4b4ca1f054b640a3794e 2023-02-12 14:55:49,164 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/336ee595d38c432bacfaeb966e909d85 2023-02-12 14:55:49,176 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212b90f9a5e99704f4c82b6509e8de430dd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230212b90f9a5e99704f4c82b6509e8de430dd 2023-02-12 14:55:49,266 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:55:49,578 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302127676f1f11353455f80789594a8961312 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302127676f1f11353455f80789594a8961312 2023-02-12 14:55:49,578 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212b52759f30c1445a0a466128c7a9e402b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230212b52759f30c1445a0a466128c7a9e402b 2023-02-12 14:55:49,581 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/84900306efbf4af5bdcfcb5e0505b16e 2023-02-12 14:55:49,584 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/b130d1bae975465db7a9a1e3b7f80591 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/b130d1bae975465db7a9a1e3b7f80591 2023-02-12 14:55:49,587 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/b130d1bae975465db7a9a1e3b7f80591, entries=4, sequenceid=13, filesize=5.4 K 2023-02-12 14:55:49,588 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/84900306efbf4af5bdcfcb5e0505b16e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/84900306efbf4af5bdcfcb5e0505b16e 2023-02-12 14:55:49,590 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/84900306efbf4af5bdcfcb5e0505b16e, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:55:49,591 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 05e3c4702d173182c255943db0b5df6e in 1253ms, sequenceid=13, compaction requested=true 2023-02-12 14:55:49,591 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:49,591 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:49,591 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:49,983 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/b669900f54724930ac779b15341612d4 2023-02-12 14:55:49,983 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/2483acfc68a446fe8f0da42229423a37 2023-02-12 14:55:49,987 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/336ee595d38c432bacfaeb966e909d85 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/336ee595d38c432bacfaeb966e909d85 2023-02-12 14:55:49,987 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/659f32103acf4b4ca1f054b640a3794e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/659f32103acf4b4ca1f054b640a3794e 2023-02-12 14:55:49,990 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/336ee595d38c432bacfaeb966e909d85, entries=4, sequenceid=13, filesize=5.4 K 2023-02-12 14:55:49,990 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/659f32103acf4b4ca1f054b640a3794e, entries=4, sequenceid=13, filesize=5.4 K 2023-02-12 14:55:49,991 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/b669900f54724930ac779b15341612d4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/b669900f54724930ac779b15341612d4 2023-02-12 14:55:49,991 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/2483acfc68a446fe8f0da42229423a37 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/2483acfc68a446fe8f0da42229423a37 2023-02-12 14:55:49,995 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/b669900f54724930ac779b15341612d4, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:55:49,996 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/2483acfc68a446fe8f0da42229423a37, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:55:49,996 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a9e7a8a10a47fb644927b2cbb23ae591 in 1658ms, sequenceid=13, compaction requested=true 2023-02-12 14:55:49,996 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for c92bf76a80608abe270eff23b9822d0c in 1658ms, sequenceid=13, compaction requested=true 2023-02-12 14:55:49,996 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:49,996 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:49,996 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:49,996 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:49,997 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:49,997 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:49,997 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:49,997 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:49,997 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-12 14:55:49,997 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-12 14:55:49,998 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:49,998 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:49,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:49,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:49,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:49,998 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:49,998 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:49,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:50,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-12 14:55:50,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f076fa6[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:50,000 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-12 14:55:50,000 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:50,001 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:50,001 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:50,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:50,001 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-12 14:55:50,001 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-12 14:55:50,001 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:50,001 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,001 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:50,002 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-12 14:55:50,002 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:50,002 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,002 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-12 14:55:50,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:50,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:50,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:50,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:50,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:50,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:50,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:50,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:50,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-12 14:55:50,005 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-12 14:55:50,005 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:50,005 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:50,005 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-12 14:55:50,005 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:50,008 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,008 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:50,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:50,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,008 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:50,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:50,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:50,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:50,008 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:50,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:50,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:50,009 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:50,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:50,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,014 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,014 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:50,014 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,015 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:50,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:50,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:50,015 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-12 14:55:50,015 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:50,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:50,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:50,015 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:50,116 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:50,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:50,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:50,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:50,118 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(8801): writing data to region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:55:50,120 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:50,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:50,123 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-12 14:55:50,123 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-12 14:55:50,123 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:50,123 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-12 14:55:50,123 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-12 14:55:50,124 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,124 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:50,125 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,125 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:50,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:50,125 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:50,125 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:50,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:50,126 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:50,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:50,126 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-12 14:55:50,126 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-12 14:55:50,126 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-12 14:55:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-12 14:55:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:50,126 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-12 14:55:50,127 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:50,127 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:50,127 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:50,127 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:50,127 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:50,127 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. started... 2023-02-12 14:55:50,127 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. started... 2023-02-12 14:55:50,127 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. started... 2023-02-12 14:55:50,127 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c92bf76a80608abe270eff23b9822d0c 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:55:50,127 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 05e3c4702d173182c255943db0b5df6e 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:55:50,127 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e7a8a10a47fb644927b2cbb23ae591 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:55:50,551 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302121be4702ebef94a2bae8a74083e060eb1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b202302121be4702ebef94a2bae8a74083e060eb1 2023-02-12 14:55:50,551 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212f4d0399e60a542438f7c446e63d1afd1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230212f4d0399e60a542438f7c446e63d1afd1 2023-02-12 14:55:50,552 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021254658e45055247f4b89ea88463e93967 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c2023021254658e45055247f4b89ea88463e93967 2023-02-12 14:55:50,958 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/1dbbb064dd6d4526af3b3e01af96fb78 2023-02-12 14:55:50,971 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/e15cfb50b9804d07b85282611e1d3932 2023-02-12 14:55:50,971 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/79eafa90a4354b66861a34ecd19a103a 2023-02-12 14:55:50,987 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212364a25485beb4046bafc1bb33b965693 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230212364a25485beb4046bafc1bb33b965693 2023-02-12 14:55:51,371 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212c227d2e87b4545a68928e45cc52ff910 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230212c227d2e87b4545a68928e45cc52ff910 2023-02-12 14:55:51,391 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212becdc102440546a3a6a9364cad40b313 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e20230212becdc102440546a3a6a9364cad40b313 2023-02-12 14:55:51,392 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/a747c49190954392893365d246ff7312 2023-02-12 14:55:51,396 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/79eafa90a4354b66861a34ecd19a103a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/79eafa90a4354b66861a34ecd19a103a 2023-02-12 14:55:51,400 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/79eafa90a4354b66861a34ecd19a103a, entries=4, sequenceid=17, filesize=5.4 K 2023-02-12 14:55:51,400 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/a747c49190954392893365d246ff7312 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/a747c49190954392893365d246ff7312 2023-02-12 14:55:51,401 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/0d0a373cb654435e9dcf0dc49904a359 2023-02-12 14:55:51,404 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/a747c49190954392893365d246ff7312, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:55:51,405 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/e15cfb50b9804d07b85282611e1d3932 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/e15cfb50b9804d07b85282611e1d3932 2023-02-12 14:55:51,405 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a9e7a8a10a47fb644927b2cbb23ae591 in 1278ms, sequenceid=17, compaction requested=true 2023-02-12 14:55:51,405 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:51,405 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:51,405 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:51,408 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/e15cfb50b9804d07b85282611e1d3932, entries=4, sequenceid=17, filesize=5.4 K 2023-02-12 14:55:51,409 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/0d0a373cb654435e9dcf0dc49904a359 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/0d0a373cb654435e9dcf0dc49904a359 2023-02-12 14:55:51,412 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/0d0a373cb654435e9dcf0dc49904a359, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:55:51,413 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for c92bf76a80608abe270eff23b9822d0c in 1286ms, sequenceid=17, compaction requested=true 2023-02-12 14:55:51,413 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:51,413 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:51,413 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:51,777 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/26a312d1c3bc447bb1f2583f8da771a9 2023-02-12 14:55:51,781 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/1dbbb064dd6d4526af3b3e01af96fb78 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/1dbbb064dd6d4526af3b3e01af96fb78 2023-02-12 14:55:51,784 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/1dbbb064dd6d4526af3b3e01af96fb78, entries=4, sequenceid=17, filesize=5.4 K 2023-02-12 14:55:51,785 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/26a312d1c3bc447bb1f2583f8da771a9 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/26a312d1c3bc447bb1f2583f8da771a9 2023-02-12 14:55:51,788 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/26a312d1c3bc447bb1f2583f8da771a9, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:55:51,789 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 05e3c4702d173182c255943db0b5df6e in 1662ms, sequenceid=17, compaction requested=true 2023-02-12 14:55:51,789 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:51,789 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:51,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:51,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:51,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-12 14:55:51,789 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-12 14:55:51,790 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,790 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:51,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:51,791 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,791 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:51,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:51,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:51,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:51,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:51,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-12 14:55:51,792 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6ce76457[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:51,792 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-12 14:55:51,792 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,794 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,795 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-12 14:55:51,795 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-12 14:55:51,795 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:51,795 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,795 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:51,796 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-12 14:55:51,796 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,796 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:51,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:51,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:51,796 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-12 14:55:51,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:51,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:51,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:51,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:51,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:51,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:51,798 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-12 14:55:51,798 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-12 14:55:51,798 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:51,798 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:51,798 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-12 14:55:51,798 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:51,799 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,799 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:51,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,800 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:51,800 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:51,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:51,801 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:51,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:51,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:51,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:51,803 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-12 14:55:51,803 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:51,803 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,803 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:51,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:51,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:51,903 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:51,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:51,914 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:51,922 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:51,930 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:51,931 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:51,957 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:55:51,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure ns:testMinorCompaction was in running list but was completed. Accepting new attempt. 2023-02-12 14:55:51,959 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure ns:testMinorCompaction 2023-02-12 14:55:51,959 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'ns:testMinorCompaction' 2023-02-12 14:55:51,959 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:51,959 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'ns:testMinorCompaction' starting 'acquire' 2023-02-12 14:55:51,959 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'ns:testMinorCompaction', kicking off acquire phase on members. 2023-02-12 14:55:51,959 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,960 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:51,961 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,961 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:51,962 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:51,962 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:51,962 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:55:51,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:51,962 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:51,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:55:51,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:51,963 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for ns:testMinorCompaction 2023-02-12 14:55:51,963 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc ns:testMinorCompaction is still present, removing 2023-02-12 14:55:51,963 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:ns:testMinorCompaction 2023-02-12 14:55:51,963 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'ns:testMinorCompaction' with timeout 60000ms 2023-02-12 14:55:51,963 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:55:51,963 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'ns:testMinorCompaction' starting 'acquire' stage 2023-02-12 14:55:51,964 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:51,964 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. started... 2023-02-12 14:55:51,964 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:55:51,964 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:55:51,964 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:51,964 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9e7a8a10a47fb644927b2cbb23ae591 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:55:51,964 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:51,964 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. started... 2023-02-12 14:55:51,964 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing c92bf76a80608abe270eff23b9822d0c 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:55:51,964 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. started... 2023-02-12 14:55:51,965 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 05e3c4702d173182c255943db0b5df6e 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:55:51,980 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/032b7e6cf7bb4dd2aa3e6e894b231de2 2023-02-12 14:55:51,981 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/69ae226ac7ba4d96955aeb6ba492375b 2023-02-12 14:55:51,984 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 032b7e6cf7bb4dd2aa3e6e894b231de2 2023-02-12 14:55:51,985 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69ae226ac7ba4d96955aeb6ba492375b 2023-02-12 14:55:51,994 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/eb74f640130d4203947954d4704ecc57 2023-02-12 14:55:51,997 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb74f640130d4203947954d4704ecc57 2023-02-12 14:55:51,997 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/032b7e6cf7bb4dd2aa3e6e894b231de2 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/032b7e6cf7bb4dd2aa3e6e894b231de2 2023-02-12 14:55:52,000 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 032b7e6cf7bb4dd2aa3e6e894b231de2 2023-02-12 14:55:52,000 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/032b7e6cf7bb4dd2aa3e6e894b231de2, entries=3, sequenceid=23, filesize=5.0 K 2023-02-12 14:55:52,001 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/eb74f640130d4203947954d4704ecc57 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/eb74f640130d4203947954d4704ecc57 2023-02-12 14:55:52,003 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eb74f640130d4203947954d4704ecc57 2023-02-12 14:55:52,003 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/eb74f640130d4203947954d4704ecc57, entries=1, sequenceid=23, filesize=5.0 K 2023-02-12 14:55:52,004 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for c92bf76a80608abe270eff23b9822d0c in 40ms, sequenceid=23, compaction requested=true 2023-02-12 14:55:52,004 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:52,004 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:52,004 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:55:52,782 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/3aed752c181b4b2d81e49f0eedfb6cb9 2023-02-12 14:55:52,786 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3aed752c181b4b2d81e49f0eedfb6cb9 2023-02-12 14:55:52,813 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/b14c2855e7484c90b8d85dbb93e0e2ba 2023-02-12 14:55:52,816 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b14c2855e7484c90b8d85dbb93e0e2ba 2023-02-12 14:55:52,816 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/69ae226ac7ba4d96955aeb6ba492375b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/69ae226ac7ba4d96955aeb6ba492375b 2023-02-12 14:55:52,820 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 69ae226ac7ba4d96955aeb6ba492375b 2023-02-12 14:55:52,820 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/69ae226ac7ba4d96955aeb6ba492375b, entries=3, sequenceid=23, filesize=5.0 K 2023-02-12 14:55:52,821 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/b14c2855e7484c90b8d85dbb93e0e2ba as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/b14c2855e7484c90b8d85dbb93e0e2ba 2023-02-12 14:55:52,824 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b14c2855e7484c90b8d85dbb93e0e2ba 2023-02-12 14:55:52,824 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/b14c2855e7484c90b8d85dbb93e0e2ba, entries=1, sequenceid=23, filesize=5.0 K 2023-02-12 14:55:52,825 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 05e3c4702d173182c255943db0b5df6e in 860ms, sequenceid=23, compaction requested=true 2023-02-12 14:55:52,825 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:52,825 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:52,825 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:55:53,597 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/8ad7f2f7e3ba4c81997162c6fd52b456 2023-02-12 14:55:53,601 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8ad7f2f7e3ba4c81997162c6fd52b456 2023-02-12 14:55:53,602 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/3aed752c181b4b2d81e49f0eedfb6cb9 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/3aed752c181b4b2d81e49f0eedfb6cb9 2023-02-12 14:55:53,605 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3aed752c181b4b2d81e49f0eedfb6cb9 2023-02-12 14:55:53,606 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/3aed752c181b4b2d81e49f0eedfb6cb9, entries=3, sequenceid=23, filesize=5.0 K 2023-02-12 14:55:53,606 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/8ad7f2f7e3ba4c81997162c6fd52b456 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/8ad7f2f7e3ba4c81997162c6fd52b456 2023-02-12 14:55:53,610 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8ad7f2f7e3ba4c81997162c6fd52b456 2023-02-12 14:55:53,610 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/8ad7f2f7e3ba4c81997162c6fd52b456, entries=1, sequenceid=23, filesize=5.0 K 2023-02-12 14:55:53,611 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a9e7a8a10a47fb644927b2cbb23ae591 in 1647ms, sequenceid=23, compaction requested=true 2023-02-12 14:55:53,611 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:53,611 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:53,611 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:55:53,611 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:55:53,611 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,611 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'ns:testMinorCompaction' locally acquired 2023-02-12 14:55:53,611 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (ns:testMinorCompaction) in zk 2023-02-12 14:55:53,612 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,612 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:53,613 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,613 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'ns:testMinorCompaction' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:55:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:53,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:53,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'ns:testMinorCompaction' on coordinator 2023-02-12 14:55:53,614 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6c314d06[Count = 0] remaining members to acquire global barrier 2023-02-12 14:55:53,614 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'ns:testMinorCompaction' starting 'in-barrier' execution. 2023-02-12 14:55:53,614 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,615 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,615 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'ns:testMinorCompaction' received 'reached' from coordinator. 2023-02-12 14:55:53,615 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'ns:testMinorCompaction' locally completed 2023-02-12 14:55:53,615 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'ns:testMinorCompaction' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:55:53,615 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,615 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:55:53,616 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'ns:testMinorCompaction' has notified controller of completion 2023-02-12 14:55:53,616 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:53,616 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,616 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'ns:testMinorCompaction' completed. 2023-02-12 14:55:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:53,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:53,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'ns:testMinorCompaction' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:55:53,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'ns:testMinorCompaction', counting down latch. Waiting for 0 more 2023-02-12 14:55:53,618 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'ns:testMinorCompaction' execution completed 2023-02-12 14:55:53,619 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:55:53,619 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:55:53,619 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:ns:testMinorCompaction 2023-02-12 14:55:53,619 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure ns:testMinorCompactionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:55:53,624 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:53,624 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:53,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:53,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:53,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:55:53,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:53,624 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:53,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:55:53,624 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:55:53,624 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:53,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:53,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:53,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:53,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:55:53,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:53,625 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:55:53,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----ns:testMinorCompaction 2023-02-12 14:55:53,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,627 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,627 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:55:53,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/ns:testMinorCompaction because node does not exist (not an error) 2023-02-12 14:55:53,627 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:55:53,627 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'ns:testMinorCompaction' 2023-02-12 14:55:53,628 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:55:53,627 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:53,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:55:53,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/ns:testMinorCompaction 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/ns:testMinorCompaction 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : ns:testMinorCompaction'' to complete. (max 300000 ms per retry) 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:55:53,628 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/ns:testMinorCompaction 2023-02-12 14:55:53,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:55:53,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:55:53,728 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:55:53,729 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:55:53,729 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:53,729 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): a9e7a8a10a47fb644927b2cbb23ae591/family1 is initiating major compaction (all files) 2023-02-12 14:55:53,730 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:53,730 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:53,730 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of a9e7a8a10a47fb644927b2cbb23ae591/family1 in ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:53,730 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/7f28702439df4e23b24224f0556a2d93, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/4bbbc69d6c7c4419980c3723d7bd92fe, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/336ee595d38c432bacfaeb966e909d85, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/79eafa90a4354b66861a34ecd19a103a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/3aed752c181b4b2d81e49f0eedfb6cb9] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp, totalSize=26.6 K 2023-02-12 14:55:53,730 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 7f28702439df4e23b24224f0556a2d93, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213744740 2023-02-12 14:55:53,731 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 4bbbc69d6c7c4419980c3723d7bd92fe, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213746541 2023-02-12 14:55:53,731 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 336ee595d38c432bacfaeb966e909d85, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213748330 2023-02-12 14:55:53,731 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 79eafa90a4354b66861a34ecd19a103a, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213750118 2023-02-12 14:55:53,732 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 3aed752c181b4b2d81e49f0eedfb6cb9, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:55:53,740 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): a9e7a8a10a47fb644927b2cbb23ae591#family1#compaction#149 average throughput is 0.83 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:54,148 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212678cddda01b44a2495c17c53cc77e023_del 2023-02-12 14:55:54,149 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212678cddda01b44a2495c17c53cc77e023_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230212678cddda01b44a2495c17c53cc77e023_del 2023-02-12 14:55:54,974 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:55:54,974 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x65f1e786 to 127.0.0.1:61565 2023-02-12 14:55:54,975 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:54,975 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:55:54,975 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x314f3d9c to 127.0.0.1:61565 2023-02-12 14:55:54,975 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:55:54,975 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48b7f37d00cc46c5a9fda5da71bb075b 2023-02-12 14:55:54,977 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family1/48b7f37d00cc46c5a9fda5da71bb075b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/48b7f37d00cc46c5a9fda5da71bb075b 2023-02-12 14:55:54,982 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 48b7f37d00cc46c5a9fda5da71bb075b 2023-02-12 14:55:54,983 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a9e7a8a10a47fb644927b2cbb23ae591/family1 of a9e7a8a10a47fb644927b2cbb23ae591 into 48b7f37d00cc46c5a9fda5da71bb075b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:55:54,983 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:54,983 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:54,983 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): a9e7a8a10a47fb644927b2cbb23ae591/family2 is initiating major compaction (all files) 2023-02-12 14:55:54,983 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:54,983 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:54,984 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of a9e7a8a10a47fb644927b2cbb23ae591/family2 in ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:55:54,984 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/aa195761aca84c9d89c1b86615b0b1f9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/b669900f54724930ac779b15341612d4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/a747c49190954392893365d246ff7312, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/8ad7f2f7e3ba4c81997162c6fd52b456] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp, totalSize=25.3 K 2023-02-12 14:55:54,984 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting aa195761aca84c9d89c1b86615b0b1f9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213744740 2023-02-12 14:55:54,985 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 0fb87bb1c7fb4a7aa2c8b67f7c412c91, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213746541 2023-02-12 14:55:54,985 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting b669900f54724930ac779b15341612d4, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213748330 2023-02-12 14:55:54,985 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting a747c49190954392893365d246ff7312, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213750118 2023-02-12 14:55:54,986 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 8ad7f2f7e3ba4c81997162c6fd52b456, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:55:54,995 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): a9e7a8a10a47fb644927b2cbb23ae591#family2#compaction#150 average throughput is 0.51 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:55,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/7f28702439df4e23b24224f0556a2d93, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/4bbbc69d6c7c4419980c3723d7bd92fe, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/336ee595d38c432bacfaeb966e909d85, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/79eafa90a4354b66861a34ecd19a103a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/3aed752c181b4b2d81e49f0eedfb6cb9] to archive 2023-02-12 14:55:55,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:55,076 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/7f28702439df4e23b24224f0556a2d93 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/7f28702439df4e23b24224f0556a2d93 2023-02-12 14:55:55,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/4bbbc69d6c7c4419980c3723d7bd92fe to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/4bbbc69d6c7c4419980c3723d7bd92fe 2023-02-12 14:55:55,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/336ee595d38c432bacfaeb966e909d85 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/336ee595d38c432bacfaeb966e909d85 2023-02-12 14:55:55,080 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/79eafa90a4354b66861a34ecd19a103a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/79eafa90a4354b66861a34ecd19a103a 2023-02-12 14:55:55,081 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/3aed752c181b4b2d81e49f0eedfb6cb9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/3aed752c181b4b2d81e49f0eedfb6cb9 2023-02-12 14:55:55,404 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212bb591f8bdc0b44b5ab3f559e0dda6b32_del 2023-02-12 14:55:55,404 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212bb591f8bdc0b44b5ab3f559e0dda6b32_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c81e728d9d4c2f636f067f89cc14862c20230212bb591f8bdc0b44b5ab3f559e0dda6b32_del 2023-02-12 14:55:55,822 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 104415834754472b9f39cd2546ca0bf9 2023-02-12 14:55:55,823 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/.tmp/family2/104415834754472b9f39cd2546ca0bf9 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/104415834754472b9f39cd2546ca0bf9 2023-02-12 14:55:55,826 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 104415834754472b9f39cd2546ca0bf9 2023-02-12 14:55:55,827 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a9e7a8a10a47fb644927b2cbb23ae591/family2 of a9e7a8a10a47fb644927b2cbb23ae591 into 104415834754472b9f39cd2546ca0bf9(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:55,827 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:55,828 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:55,828 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 05e3c4702d173182c255943db0b5df6e/family1 is initiating major compaction (all files) 2023-02-12 14:55:55,828 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:55,828 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:55,828 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 05e3c4702d173182c255943db0b5df6e/family1 in ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:55,828 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/928d9bed69fb4d249b5091aeaa1234e0, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/2d50c2bae4214877b9a3010fbd9a2841, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/b130d1bae975465db7a9a1e3b7f80591, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/1dbbb064dd6d4526af3b3e01af96fb78, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/69ae226ac7ba4d96955aeb6ba492375b] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp, totalSize=26.6 K 2023-02-12 14:55:55,829 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 928d9bed69fb4d249b5091aeaa1234e0, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213744740 2023-02-12 14:55:55,829 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 2d50c2bae4214877b9a3010fbd9a2841, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213746541 2023-02-12 14:55:55,829 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting b130d1bae975465db7a9a1e3b7f80591, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213748330 2023-02-12 14:55:55,830 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 1dbbb064dd6d4526af3b3e01af96fb78, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213750118 2023-02-12 14:55:55,830 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 69ae226ac7ba4d96955aeb6ba492375b, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:55:55,844 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 05e3c4702d173182c255943db0b5df6e#family1#compaction#151 average throughput is 0.24 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:55,852 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021228ffebf5cdb94913a0257c0ee65b2de5_del 2023-02-12 14:55:55,853 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021228ffebf5cdb94913a0257c0ee65b2de5_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021228ffebf5cdb94913a0257c0ee65b2de5_del 2023-02-12 14:55:55,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/aa195761aca84c9d89c1b86615b0b1f9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/b669900f54724930ac779b15341612d4, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/a747c49190954392893365d246ff7312, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/8ad7f2f7e3ba4c81997162c6fd52b456] to archive 2023-02-12 14:55:55,873 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:55,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/aa195761aca84c9d89c1b86615b0b1f9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/aa195761aca84c9d89c1b86615b0b1f9 2023-02-12 14:55:55,876 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/0fb87bb1c7fb4a7aa2c8b67f7c412c91 2023-02-12 14:55:55,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/b669900f54724930ac779b15341612d4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/b669900f54724930ac779b15341612d4 2023-02-12 14:55:55,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/a747c49190954392893365d246ff7312 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/a747c49190954392893365d246ff7312 2023-02-12 14:55:55,880 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/8ad7f2f7e3ba4c81997162c6fd52b456 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family2/8ad7f2f7e3ba4c81997162c6fd52b456 2023-02-12 14:55:56,268 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 129cca3d69ed4ecf880fda28d500be5b 2023-02-12 14:55:56,269 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family1/129cca3d69ed4ecf880fda28d500be5b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/129cca3d69ed4ecf880fda28d500be5b 2023-02-12 14:55:56,273 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 129cca3d69ed4ecf880fda28d500be5b 2023-02-12 14:55:56,274 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 05e3c4702d173182c255943db0b5df6e/family1 of 05e3c4702d173182c255943db0b5df6e into 129cca3d69ed4ecf880fda28d500be5b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:56,274 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:56,274 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:56,274 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 05e3c4702d173182c255943db0b5df6e/family2 is initiating major compaction (all files) 2023-02-12 14:55:56,274 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:56,275 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:56,275 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 05e3c4702d173182c255943db0b5df6e/family2 in ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:55:56,275 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/2d78fed16b004973b6581a70b2280257, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/81ce6782b8f74be8909850afec94668b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/84900306efbf4af5bdcfcb5e0505b16e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/26a312d1c3bc447bb1f2583f8da771a9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/b14c2855e7484c90b8d85dbb93e0e2ba] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp, totalSize=25.3 K 2023-02-12 14:55:56,275 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 2d78fed16b004973b6581a70b2280257, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213744740 2023-02-12 14:55:56,276 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 81ce6782b8f74be8909850afec94668b, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213746541 2023-02-12 14:55:56,276 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 84900306efbf4af5bdcfcb5e0505b16e, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213748330 2023-02-12 14:55:56,277 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 26a312d1c3bc447bb1f2583f8da771a9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213750118 2023-02-12 14:55:56,277 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting b14c2855e7484c90b8d85dbb93e0e2ba, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:55:56,279 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/928d9bed69fb4d249b5091aeaa1234e0, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/2d50c2bae4214877b9a3010fbd9a2841, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/b130d1bae975465db7a9a1e3b7f80591, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/1dbbb064dd6d4526af3b3e01af96fb78, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/69ae226ac7ba4d96955aeb6ba492375b] to archive 2023-02-12 14:55:56,280 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:56,282 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/928d9bed69fb4d249b5091aeaa1234e0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/928d9bed69fb4d249b5091aeaa1234e0 2023-02-12 14:55:56,283 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/2d50c2bae4214877b9a3010fbd9a2841 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/2d50c2bae4214877b9a3010fbd9a2841 2023-02-12 14:55:56,284 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/b130d1bae975465db7a9a1e3b7f80591 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/b130d1bae975465db7a9a1e3b7f80591 2023-02-12 14:55:56,286 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/1dbbb064dd6d4526af3b3e01af96fb78 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/1dbbb064dd6d4526af3b3e01af96fb78 2023-02-12 14:55:56,289 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/69ae226ac7ba4d96955aeb6ba492375b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/69ae226ac7ba4d96955aeb6ba492375b 2023-02-12 14:55:56,300 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 05e3c4702d173182c255943db0b5df6e#family2#compaction#152 average throughput is 0.08 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:56,318 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b20230212e6e38ef1486949cd94979d067ebc90a7_del 2023-02-12 14:55:56,318 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212e6e38ef1486949cd94979d067ebc90a7_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/c4ca4238a0b923820dcc509a6f75849b20230212e6e38ef1486949cd94979d067ebc90a7_del 2023-02-12 14:55:56,755 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e434ced0677f4734a261d48b82235c0d 2023-02-12 14:55:56,756 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/.tmp/family2/e434ced0677f4734a261d48b82235c0d as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/e434ced0677f4734a261d48b82235c0d 2023-02-12 14:55:56,760 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e434ced0677f4734a261d48b82235c0d 2023-02-12 14:55:56,761 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 05e3c4702d173182c255943db0b5df6e/family2 of 05e3c4702d173182c255943db0b5df6e into e434ced0677f4734a261d48b82235c0d(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:56,762 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:56,762 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:56,762 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): c92bf76a80608abe270eff23b9822d0c/family1 is initiating major compaction (all files) 2023-02-12 14:55:56,762 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:56,762 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:56,763 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of c92bf76a80608abe270eff23b9822d0c/family1 in ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:56,763 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/bfc399a98594442780869197a4dd007c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/244da5eeee364f48921ba11bf01995b5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/659f32103acf4b4ca1f054b640a3794e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/e15cfb50b9804d07b85282611e1d3932, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/032b7e6cf7bb4dd2aa3e6e894b231de2] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp, totalSize=26.6 K 2023-02-12 14:55:56,764 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting bfc399a98594442780869197a4dd007c, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213744739 2023-02-12 14:55:56,764 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 244da5eeee364f48921ba11bf01995b5, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213746541 2023-02-12 14:55:56,764 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 659f32103acf4b4ca1f054b640a3794e, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213748330 2023-02-12 14:55:56,765 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting e15cfb50b9804d07b85282611e1d3932, keycount=4, bloomtype=ROW, size=5.4 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213750117 2023-02-12 14:55:56,765 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 032b7e6cf7bb4dd2aa3e6e894b231de2, keycount=3, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:55:56,770 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/2d78fed16b004973b6581a70b2280257, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/81ce6782b8f74be8909850afec94668b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/84900306efbf4af5bdcfcb5e0505b16e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/26a312d1c3bc447bb1f2583f8da771a9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/b14c2855e7484c90b8d85dbb93e0e2ba] to archive 2023-02-12 14:55:56,771 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:56,772 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/2d78fed16b004973b6581a70b2280257 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/2d78fed16b004973b6581a70b2280257 2023-02-12 14:55:56,774 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/81ce6782b8f74be8909850afec94668b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/81ce6782b8f74be8909850afec94668b 2023-02-12 14:55:56,775 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/84900306efbf4af5bdcfcb5e0505b16e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/84900306efbf4af5bdcfcb5e0505b16e 2023-02-12 14:55:56,776 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/26a312d1c3bc447bb1f2583f8da771a9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/26a312d1c3bc447bb1f2583f8da771a9 2023-02-12 14:55:56,780 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/b14c2855e7484c90b8d85dbb93e0e2ba to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family2/b14c2855e7484c90b8d85dbb93e0e2ba 2023-02-12 14:55:56,790 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): c92bf76a80608abe270eff23b9822d0c#family1#compaction#153 average throughput is 0.12 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:56,803 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302128fc7380c31244135941144d4df31e1a4_del 2023-02-12 14:55:56,803 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302128fc7380c31244135941144d4df31e1a4_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e202302128fc7380c31244135941144d4df31e1a4_del 2023-02-12 14:55:57,621 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1111cb1cbdd54fefaf69fd0fff78e826 2023-02-12 14:55:57,622 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family1/1111cb1cbdd54fefaf69fd0fff78e826 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/1111cb1cbdd54fefaf69fd0fff78e826 2023-02-12 14:55:57,626 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1111cb1cbdd54fefaf69fd0fff78e826 2023-02-12 14:55:57,628 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c92bf76a80608abe270eff23b9822d0c/family1 of c92bf76a80608abe270eff23b9822d0c into 1111cb1cbdd54fefaf69fd0fff78e826(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:57,628 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:57,628 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:55:57,628 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): c92bf76a80608abe270eff23b9822d0c/family2 is initiating major compaction (all files) 2023-02-12 14:55:57,628 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:55:57,632 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:55:57,632 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of c92bf76a80608abe270eff23b9822d0c/family2 in ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:55:57,632 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/f893f2d847e0428cb10bfcbfb20365b6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/2483acfc68a446fe8f0da42229423a37, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/0d0a373cb654435e9dcf0dc49904a359, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/eb74f640130d4203947954d4704ecc57] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp, totalSize=25.3 K 2023-02-12 14:55:57,633 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f893f2d847e0428cb10bfcbfb20365b6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213744739 2023-02-12 14:55:57,634 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting e8494d8bd0ef4e7d9c881a9a0252d0a9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213746541 2023-02-12 14:55:57,634 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 2483acfc68a446fe8f0da42229423a37, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213748330 2023-02-12 14:55:57,635 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 0d0a373cb654435e9dcf0dc49904a359, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213750117 2023-02-12 14:55:57,635 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting eb74f640130d4203947954d4704ecc57, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:55:57,645 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): c92bf76a80608abe270eff23b9822d0c#family2#compaction#154 average throughput is 0.34 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:55:57,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/bfc399a98594442780869197a4dd007c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/244da5eeee364f48921ba11bf01995b5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/659f32103acf4b4ca1f054b640a3794e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/e15cfb50b9804d07b85282611e1d3932, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/032b7e6cf7bb4dd2aa3e6e894b231de2] to archive 2023-02-12 14:55:57,671 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:57,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/bfc399a98594442780869197a4dd007c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/bfc399a98594442780869197a4dd007c 2023-02-12 14:55:57,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/244da5eeee364f48921ba11bf01995b5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/244da5eeee364f48921ba11bf01995b5 2023-02-12 14:55:57,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/659f32103acf4b4ca1f054b640a3794e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/659f32103acf4b4ca1f054b640a3794e 2023-02-12 14:55:57,677 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/e15cfb50b9804d07b85282611e1d3932 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/e15cfb50b9804d07b85282611e1d3932 2023-02-12 14:55:57,678 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/032b7e6cf7bb4dd2aa3e6e894b231de2 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/032b7e6cf7bb4dd2aa3e6e894b231de2 2023-02-12 14:55:58,057 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302122403c079c56145a39aafc96949b299a8_del 2023-02-12 14:55:58,057 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302122403c079c56145a39aafc96949b299a8_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family2/d41d8cd98f00b204e9800998ecf8427e202302122403c079c56145a39aafc96949b299a8_del 2023-02-12 14:55:58,473 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ce76e71c9c448b8175169bcfb993bc 2023-02-12 14:55:58,474 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/.tmp/family2/c4ce76e71c9c448b8175169bcfb993bc as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/c4ce76e71c9c448b8175169bcfb993bc 2023-02-12 14:55:58,478 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ce76e71c9c448b8175169bcfb993bc 2023-02-12 14:55:58,480 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in c92bf76a80608abe270eff23b9822d0c/family2 of c92bf76a80608abe270eff23b9822d0c into c4ce76e71c9c448b8175169bcfb993bc(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:55:58,480 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:58,484 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:58,487 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:58,491 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:58,491 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:58,497 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-12 14:55:58,511 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(306): The compaction type is PART_FILES, the request has 3 del files, 3 selected files, and 0 irrelevant files table 'ns:testMinorCompaction' and column 'family1' 2023-02-12 14:55:58,514 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302128fc7380c31244135941144d4df31e1a4_del 2023-02-12 14:55:58,517 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b2023021228ffebf5cdb94913a0257c0ee65b2de5_del 2023-02-12 14:55:58,520 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212678cddda01b44a2495c17c53cc77e023_del 2023-02-12 14:55:58,520 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:58,520 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(139): Connect 0x44f9d106 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:55:58,524 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@8d46794, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:55:58,526 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230212 for table='ns:testMinorCompaction' and column='family1' 2023-02-12 14:55:58,526 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230212 for table='ns:testMinorCompaction' and column='family1' 2023-02-12 14:55:58,526 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230212 for table='ns:testMinorCompaction' and column='family1' 2023-02-12 14:55:58,575 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/f893f2d847e0428cb10bfcbfb20365b6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/2483acfc68a446fe8f0da42229423a37, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/0d0a373cb654435e9dcf0dc49904a359, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/eb74f640130d4203947954d4704ecc57] to archive 2023-02-12 14:55:58,576 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:58,578 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/f893f2d847e0428cb10bfcbfb20365b6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/f893f2d847e0428cb10bfcbfb20365b6 2023-02-12 14:55:58,579 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/e8494d8bd0ef4e7d9c881a9a0252d0a9 2023-02-12 14:55:58,580 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/2483acfc68a446fe8f0da42229423a37 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/2483acfc68a446fe8f0da42229423a37 2023-02-12 14:55:58,581 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/0d0a373cb654435e9dcf0dc49904a359 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/0d0a373cb654435e9dcf0dc49904a359 2023-02-12 14:55:58,582 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/eb74f640130d4203947954d4704ecc57 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family2/eb74f640130d4203947954d4704ecc57 2023-02-12 14:55:58,966 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212295de38ac9f445dba5406f66e29962c7 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230212295de38ac9f445dba5406f66e29962c7 2023-02-12 14:55:58,967 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:58,967 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302126570423468aa4dc9a638cc61a36e7213 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c202302126570423468aa4dc9a638cc61a36e7213 2023-02-12 14:55:58,968 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:58,987 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:55:58,990 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:60952, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:55:58,996 DEBUG [MobFileCompactionChore-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:55:59,000 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:50952, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:55:59,012 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230212/family1/a34a3859f0264482ad7818714b9319a0 first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x01] 2023-02-12 14:55:59,012 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/family1/6facd801448644228c742643ed3f93fd first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x01] 2023-02-12 14:55:59,013 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230212/family1/a34a3859f0264482ad7818714b9319a0}] 2023-02-12 14:55:59,013 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/family1/6facd801448644228c742643ed3f93fd}] 2023-02-12 14:55:59,022 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230212/family1/a34a3859f0264482ad7818714b9319a0 for inclusion in a9e7a8a10a47fb644927b2cbb23ae591/family1 2023-02-12 14:55:59,022 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/family1/6facd801448644228c742643ed3f93fd for inclusion in c92bf76a80608abe270eff23b9822d0c/family1 2023-02-12 14:55:59,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x01 2023-02-12 14:55:59,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x01 2023-02-12 14:55:59,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last=1 2023-02-12 14:55:59,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-12 14:55:59,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for c92bf76a80608abe270eff23b9822d0c: 2023-02-12 14:55:59,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for a9e7a8a10a47fb644927b2cbb23ae591: 2023-02-12 14:55:59,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230212/family1/6facd801448644228c742643ed3f93fd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__f0fldhvc7kbgph06d35dj1d84stob16sgo3ep3ikusujig7rdhr1d9su3sgs52au/family1/6facd801448644228c742643ed3f93fd 2023-02-12 14:55:59,025 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230212/family1/a34a3859f0264482ad7818714b9319a0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__287ntm3k2m99jh7svkud3ci0rnbjfdvqra6vr8cre9j6h7ta1dm8er5j1nhrfuaq/family1/a34a3859f0264482ad7818714b9319a0 2023-02-12 14:55:59,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__f0fldhvc7kbgph06d35dj1d84stob16sgo3ep3ikusujig7rdhr1d9su3sgs52au/family1/6facd801448644228c742643ed3f93fd as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/00fed86fc6e3464586b05ac3eacf26b9_SeqId_28_ 2023-02-12 14:55:59,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__287ntm3k2m99jh7svkud3ci0rnbjfdvqra6vr8cre9j6h7ta1dm8er5j1nhrfuaq/family1/a34a3859f0264482ad7818714b9319a0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/5af29f1929c4414cb02806b5f38592c4_SeqId_28_ 2023-02-12 14:55:59,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__f0fldhvc7kbgph06d35dj1d84stob16sgo3ep3ikusujig7rdhr1d9su3sgs52au/family1/6facd801448644228c742643ed3f93fd into c92bf76a80608abe270eff23b9822d0c/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/00fed86fc6e3464586b05ac3eacf26b9_SeqId_28_ - updating store file list. 2023-02-12 14:55:59,029 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__287ntm3k2m99jh7svkud3ci0rnbjfdvqra6vr8cre9j6h7ta1dm8er5j1nhrfuaq/family1/a34a3859f0264482ad7818714b9319a0 into a9e7a8a10a47fb644927b2cbb23ae591/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/5af29f1929c4414cb02806b5f38592c4_SeqId_28_ - updating store file list. 2023-02-12 14:55:59,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/00fed86fc6e3464586b05ac3eacf26b9_SeqId_28_ into c92bf76a80608abe270eff23b9822d0c/family1 2023-02-12 14:55:59,033 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__f0fldhvc7kbgph06d35dj1d84stob16sgo3ep3ikusujig7rdhr1d9su3sgs52au/family1/6facd801448644228c742643ed3f93fd into c92bf76a80608abe270eff23b9822d0c/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/family1/00fed86fc6e3464586b05ac3eacf26b9_SeqId_28_) 2023-02-12 14:55:59,033 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/5af29f1929c4414cb02806b5f38592c4_SeqId_28_ into a9e7a8a10a47fb644927b2cbb23ae591/family1 2023-02-12 14:55:59,033 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__287ntm3k2m99jh7svkud3ci0rnbjfdvqra6vr8cre9j6h7ta1dm8er5j1nhrfuaq/family1/a34a3859f0264482ad7818714b9319a0 into a9e7a8a10a47fb644927b2cbb23ae591/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/family1/5af29f1929c4414cb02806b5f38592c4_SeqId_28_) 2023-02-12 14:55:59,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__f0fldhvc7kbgph06d35dj1d84stob16sgo3ep3ikusujig7rdhr1d9su3sgs52au/family1/6facd801448644228c742643ed3f93fd 2023-02-12 14:55:59,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__287ntm3k2m99jh7svkud3ci0rnbjfdvqra6vr8cre9j6h7ta1dm8er5j1nhrfuaq/family1/a34a3859f0264482ad7818714b9319a0 2023-02-12 14:55:59,035 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:59,036 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:59,036 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:59,037 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:59,037 DEBUG [MobFileCompactionChore-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230212e145c87a0e4f48f9a49a8dd316808056 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/d41d8cd98f00b204e9800998ecf8427e20230212e145c87a0e4f48f9a49a8dd316808056 2023-02-12 14:55:59,037 INFO [MobFileCompactionChore-pool-0] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-12 14:55:59,038 DEBUG [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/d41d8cd98f00b204e9800998ecf8427e20230212' 2023-02-12 14:55:59,038 INFO [MobFileCompactionChore-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230212 for table='ns:testMinorCompaction' and column='family1' 2023-02-12 14:55:59,039 DEBUG [MobFileCompactionChore-pool-2] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230212fe1e5590b3764abd856ab75065de672a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c81e728d9d4c2f636f067f89cc14862c20230212fe1e5590b3764abd856ab75065de672a 2023-02-12 14:55:59,039 INFO [MobFileCompactionChore-pool-2] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-12 14:55:59,039 DEBUG [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c81e728d9d4c2f636f067f89cc14862c20230212' 2023-02-12 14:55:59,039 INFO [MobFileCompactionChore-pool-2] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230212 for table='ns:testMinorCompaction' and column='family1' 2023-02-12 14:55:59,358 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212e39ee9ef59e1474cbb3541f65cee9c9f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b20230212e39ee9ef59e1474cbb3541f65cee9c9f 2023-02-12 14:55:59,359 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:59,388 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230212/family1/a8130255ae4c403d9a125882b41b0e71 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x01] 2023-02-12 14:55:59,388 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230212/family1/a8130255ae4c403d9a125882b41b0e71}] 2023-02-12 14:55:59,390 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230212/family1/a8130255ae4c403d9a125882b41b0e71 for inclusion in 05e3c4702d173182c255943db0b5df6e/family1 2023-02-12 14:55:59,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x01 2023-02-12 14:55:59,392 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-12 14:55:59,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 05e3c4702d173182c255943db0b5df6e: 2023-02-12 14:55:59,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230212/family1/a8130255ae4c403d9a125882b41b0e71 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__lr9mq9r83dcje3ucni6up5f93c7dp2lnqfnll0djjlh70j41nd9gdgjd8tduhmns/family1/a8130255ae4c403d9a125882b41b0e71 2023-02-12 14:55:59,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__lr9mq9r83dcje3ucni6up5f93c7dp2lnqfnll0djjlh70j41nd9gdgjd8tduhmns/family1/a8130255ae4c403d9a125882b41b0e71 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/914ed9be1ce44f33a25ff0432a8ea6c5_SeqId_28_ 2023-02-12 14:55:59,395 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__lr9mq9r83dcje3ucni6up5f93c7dp2lnqfnll0djjlh70j41nd9gdgjd8tduhmns/family1/a8130255ae4c403d9a125882b41b0e71 into 05e3c4702d173182c255943db0b5df6e/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/914ed9be1ce44f33a25ff0432a8ea6c5_SeqId_28_ - updating store file list. 2023-02-12 14:55:59,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/914ed9be1ce44f33a25ff0432a8ea6c5_SeqId_28_ into 05e3c4702d173182c255943db0b5df6e/family1 2023-02-12 14:55:59,398 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__lr9mq9r83dcje3ucni6up5f93c7dp2lnqfnll0djjlh70j41nd9gdgjd8tduhmns/family1/a8130255ae4c403d9a125882b41b0e71 into 05e3c4702d173182c255943db0b5df6e/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/family1/914ed9be1ce44f33a25ff0432a8ea6c5_SeqId_28_) 2023-02-12 14:55:59,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__ns_testMinorCompaction__lr9mq9r83dcje3ucni6up5f93c7dp2lnqfnll0djjlh70j41nd9gdgjd8tduhmns/family1/a8130255ae4c403d9a125882b41b0e71 2023-02-12 14:55:59,400 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:59,400 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:55:59,402 DEBUG [MobFileCompactionChore-pool-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021222d5c3e9b9c246f388ee42e7e371681b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/ns/testMinorCompaction/eb474a9c8cfe32571864cdae147b59d3/family1/c4ca4238a0b923820dcc509a6f75849b2023021222d5c3e9b9c246f388ee42e7e371681b 2023-02-12 14:55:59,402 INFO [MobFileCompactionChore-pool-1] mob.MobUtils(508): Table ns:testMinorCompaction 1 expired mob files are deleted 2023-02-12 14:55:59,402 DEBUG [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/ns/testMinorCompaction/c4ca4238a0b923820dcc509a6f75849b20230212' 2023-02-12 14:55:59,402 INFO [MobFileCompactionChore-pool-1] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230212 for table='ns:testMinorCompaction' and column='family1' 2023-02-12 14:55:59,402 INFO [Listener at localhost/37945] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='ns:testMinorCompaction' column='family1' 2023-02-12 14:55:59,404 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:59,407 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:59,410 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:55:59,410 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:55:59,431 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompaction Thread=251 (was 284), OpenFileDescriptor=631 (was 761), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=198 (was 122) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=3829 (was 3700) - AvailableMemoryMB LEAK? - 2023-02-12 14:55:59,449 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=251, OpenFileDescriptor=631, MaxFileDescriptor=60000, SystemLoadAverage=198, ProcessCount=168, AvailableMemoryMB=3828 2023-02-12 14:55:59,451 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} 2023-02-12 14:55:59,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy 2023-02-12 14:55:59,453 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:55:59,453 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMinorCompactionWithMonthlyPolicy" procId is: 112 2023-02-12 14:55:59,454 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:55:59,454 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-12 14:55:59,455 DEBUG [HFileArchiver-19] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:55:59,455 DEBUG [HFileArchiver-19] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff empty. 2023-02-12 14:55:59,456 DEBUG [HFileArchiver-19] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:55:59,456 DEBUG [PEWorker-5] procedure.DeleteTableProcedure(328): Archived testMinorCompactionWithMonthlyPolicy regions 2023-02-12 14:55:59,555 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-12 14:55:59,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-12 14:55:59,863 DEBUG [PEWorker-5] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMinorCompactionWithMonthlyPolicy/.tabledesc/.tableinfo.0000000001 2023-02-12 14:55:59,864 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(7671): creating {ENCODED => 98f05e4653d1b15554e6b39d5c185fff, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testMinorCompactionWithMonthlyPolicy', {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:56:00,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-12 14:56:00,269 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:00,269 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1603): Closing 98f05e4653d1b15554e6b39d5c185fff, disabling compactions & flushes 2023-02-12 14:56:00,269 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:00,269 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:00,269 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. after waiting 0 ms 2023-02-12 14:56:00,269 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:00,269 INFO [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:00,269 DEBUG [RegionOpenAndInit-testMinorCompactionWithMonthlyPolicy-pool-0] regionserver.HRegion(1557): Region close journal for 98f05e4653d1b15554e6b39d5c185fff: Waiting for close lock at 1676213760269Disabling compacts and flushes for region at 1676213760269Disabling writes for close at 1676213760269Writing region close event to WAL at 1676213760269Closed at 1676213760269 2023-02-12 14:56:00,271 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:56:00,272 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676213760272"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213760272"}]},"ts":"1676213760272"} 2023-02-12 14:56:00,274 INFO [PEWorker-5] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-12 14:56:00,274 INFO [PEWorker-5] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:56:00,274 DEBUG [PEWorker-5] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213760274"}]},"ts":"1676213760274"} 2023-02-12 14:56:00,275 INFO [PEWorker-5] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLING in hbase:meta 2023-02-12 14:56:00,278 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=98f05e4653d1b15554e6b39d5c185fff, ASSIGN}] 2023-02-12 14:56:00,280 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=98f05e4653d1b15554e6b39d5c185fff, ASSIGN 2023-02-12 14:56:00,280 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=98f05e4653d1b15554e6b39d5c185fff, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:56:00,431 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=98f05e4653d1b15554e6b39d5c185fff, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:00,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676213760431"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213760431"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213760431"}]},"ts":"1676213760431"} 2023-02-12 14:56:00,433 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; OpenRegionProcedure 98f05e4653d1b15554e6b39d5c185fff, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:56:00,558 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-12 14:56:00,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:00,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 98f05e4653d1b15554e6b39d5c185fff, NAME => 'testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff.', STARTKEY => '', ENDKEY => ''} 2023-02-12 14:56:00,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMinorCompactionWithMonthlyPolicy 98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:00,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:00,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:00,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:00,589 INFO [StoreOpener-98f05e4653d1b15554e6b39d5c185fff-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:00,590 DEBUG [StoreOpener-98f05e4653d1b15554e6b39d5c185fff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:00,590 DEBUG [StoreOpener-98f05e4653d1b15554e6b39d5c185fff-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:00,590 INFO [StoreOpener-98f05e4653d1b15554e6b39d5c185fff-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 98f05e4653d1b15554e6b39d5c185fff columnFamilyName family1 2023-02-12 14:56:00,592 INFO [StoreOpener-98f05e4653d1b15554e6b39d5c185fff-1] regionserver.HStore(310): Store=98f05e4653d1b15554e6b39d5c185fff/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:56:00,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:00,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:00,595 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:00,597 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:56:00,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 98f05e4653d1b15554e6b39d5c185fff; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60123372, jitterRate=-0.10409194231033325}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-12 14:56:00,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 98f05e4653d1b15554e6b39d5c185fff: Running coprocessor pre-open hook at 1676213760588Writing region info on filesystem at 1676213760588Initializing all the Stores at 1676213760588Instantiating store for column family {NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'} at 1676213760589 (+1 ms)Cleaning up temporary data from old regions at 1676213760593 (+4 ms)Cleaning up detritus from prior splits at 1676213760593Running coprocessor post-open hooks at 1676213760598 (+5 ms)Region opened successfully at 1676213760598 2023-02-12 14:56:00,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff., pid=114, masterSystemTime=1676213760585 2023-02-12 14:56:00,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:00,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:00,600 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=98f05e4653d1b15554e6b39d5c185fff, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:00,600 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676213760600"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213760600"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213760600"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213760600"}]},"ts":"1676213760600"} 2023-02-12 14:56:00,602 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-12 14:56:00,602 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; OpenRegionProcedure 98f05e4653d1b15554e6b39d5c185fff, server=jenkins-hbase4.apache.org,45319,1676213617001 in 168 msec 2023-02-12 14:56:00,603 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-12 14:56:00,603 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testMinorCompactionWithMonthlyPolicy, region=98f05e4653d1b15554e6b39d5c185fff, ASSIGN in 324 msec 2023-02-12 14:56:00,604 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:56:00,604 DEBUG [PEWorker-4] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMinorCompactionWithMonthlyPolicy","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213760604"}]},"ts":"1676213760604"} 2023-02-12 14:56:00,605 INFO [PEWorker-4] hbase.MetaTableAccessor(1635): Updated tableName=testMinorCompactionWithMonthlyPolicy, state=ENABLED in hbase:meta 2023-02-12 14:56:00,608 INFO [PEWorker-4] procedure.CreateTableProcedure(80): pid=112, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:56:00,609 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; CreateTableProcedure table=testMinorCompactionWithMonthlyPolicy in 1.1580 sec 2023-02-12 14:56:01,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=112 2023-02-12 14:56:01,560 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMinorCompactionWithMonthlyPolicy, procId: 112 completed 2023-02-12 14:56:01,577 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:01,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,581 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:01,581 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:01,581 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:01,581 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:01,582 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,582 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,583 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:01,583 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:01,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:01,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:01,583 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:01,583 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,584 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:01,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,584 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,585 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:01,585 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:01,585 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:01,585 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:01,586 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=74.11 KB heapSize=148.69 KB 2023-02-12 14:56:02,004 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151130edd1661ed29e4ea0a349ea3f6f1687c4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130edd1661ed29e4ea0a349ea3f6f1687c4 2023-02-12 14:56:02,136 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:02,408 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=74.1 K, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/eb69b5ec7e2140f081f7b84f58ba40ab 2023-02-12 14:56:02,412 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/eb69b5ec7e2140f081f7b84f58ba40ab as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/eb69b5ec7e2140f081f7b84f58ba40ab 2023-02-12 14:56:02,415 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/eb69b5ec7e2140f081f7b84f58ba40ab, entries=1000, sequenceid=5, filesize=166.0 K 2023-02-12 14:56:02,415 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74.11 KB/75890, heapSize ~148.67 KB/152240, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 829ms, sequenceid=5, compaction requested=false 2023-02-12 14:56:02,416 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:02,416 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:02,417 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:02,418 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,418 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:02,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:02,418 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,419 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:02,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:02,420 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7ea15d19[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:02,420 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:02,420 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,421 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:02,421 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:02,421 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,421 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:02,422 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,422 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:02,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:02,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:02,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:02,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:02,425 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:02,425 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:02,425 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:02,425 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,425 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:02,427 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,427 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:02,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,427 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:02,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:02,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:02,428 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:02,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,429 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:02,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:02,431 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:02,431 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:56:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:02,431 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:02,431 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:02,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:02,431 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:02,532 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:02,532 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:02,536 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:02,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:02,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,538 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:02,538 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:02,538 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:02,538 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:02,538 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,538 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,539 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,539 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:02,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:02,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:02,540 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:02,540 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:02,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,540 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:02,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,541 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,541 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:02,541 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:02,541 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:02,541 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:02,541 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:02,541 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:02,541 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:02,541 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:02,541 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:02,541 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=74 B heapSize=408 B 2023-02-12 14:56:02,957 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151128b42e87e5a55b45b9ba6e857bd51ee3da to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b42e87e5a55b45b9ba6e857bd51ee3da 2023-02-12 14:56:03,362 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=74, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/e917cb8429bf49d7915b157e57e91070 2023-02-12 14:56:03,366 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/e917cb8429bf49d7915b157e57e91070 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e917cb8429bf49d7915b157e57e91070 2023-02-12 14:56:03,369 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e917cb8429bf49d7915b157e57e91070, entries=1, sequenceid=9, filesize=5.0 K 2023-02-12 14:56:03,370 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~74 B/74, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 829ms, sequenceid=9, compaction requested=false 2023-02-12 14:56:03,370 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:03,370 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:03,370 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:03,370 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:03,370 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,370 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:03,370 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:03,371 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,371 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:03,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:03,372 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,372 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:03,373 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:03,373 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7a5f7734[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:03,373 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:03,373 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,375 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,375 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,375 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:03,375 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:03,375 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:03,375 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,375 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:03,376 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:03,376 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:03,376 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,376 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:03,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:03,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:03,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:03,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:03,379 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:03,379 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:03,379 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:03,379 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,379 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:03,380 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,380 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,380 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:03,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:03,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:03,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:03,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:03,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:03,381 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,382 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:03,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,384 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,384 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:03,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:03,384 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:56:03,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,384 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:03,384 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:03,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:03,485 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:03,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:03,489 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:03,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:03,491 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,491 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:03,491 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:03,491 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:03,491 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:03,491 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,491 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,497 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:03,497 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:03,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:03,497 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:03,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,497 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:03,498 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:03,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,498 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,498 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:03,498 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:03,498 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:03,499 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:03,499 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:03,499 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:03,909 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512014d34e344d0594caf818427381041d244 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512014d34e344d0594caf818427381041d244 2023-02-12 14:56:04,314 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/e74ff128405340f892d6d954a99e6325 2023-02-12 14:56:04,318 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/e74ff128405340f892d6d954a99e6325 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e74ff128405340f892d6d954a99e6325 2023-02-12 14:56:04,321 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e74ff128405340f892d6d954a99e6325, entries=1, sequenceid=13, filesize=4.9 K 2023-02-12 14:56:04,321 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 822ms, sequenceid=13, compaction requested=true 2023-02-12 14:56:04,321 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:04,322 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:04,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:04,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:04,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:04,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:04,323 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,323 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:04,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:04,324 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,324 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:04,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:04,325 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2dcd71f4[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:04,325 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:04,325 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,326 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,326 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:04,326 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:04,326 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:04,326 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,326 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:04,328 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:04,328 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:04,328 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,328 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:04,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:04,330 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:04,330 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:04,331 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:04,331 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:04,331 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,331 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:04,332 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:04,332 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,332 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:04,332 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:04,333 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,333 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:04,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,334 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:04,336 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:04,336 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:04,336 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:04,336 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:04,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,337 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,436 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:04,437 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:04,441 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:04,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:04,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,442 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:04,442 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:04,442 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:04,442 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:04,443 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,443 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,444 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,444 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:04,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:04,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:04,444 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:04,444 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:04,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:04,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,445 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:04,445 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:04,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:04,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:04,445 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:04,446 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:04,446 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:04,446 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:04,446 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:04,446 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:04,856 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151205de31463a6c1643fb8e0d1e69348c8922 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205de31463a6c1643fb8e0d1e69348c8922 2023-02-12 14:56:05,262 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/5a0af93be5284cb782a470201c64e51b 2023-02-12 14:56:05,267 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/5a0af93be5284cb782a470201c64e51b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/5a0af93be5284cb782a470201c64e51b 2023-02-12 14:56:05,270 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/5a0af93be5284cb782a470201c64e51b, entries=1, sequenceid=17, filesize=4.9 K 2023-02-12 14:56:05,271 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 825ms, sequenceid=17, compaction requested=true 2023-02-12 14:56:05,271 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:05,271 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:05,271 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:05,274 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,274 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:05,275 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,275 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:05,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:05,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:05,276 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b724036[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:05,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:05,276 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,277 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,277 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:05,277 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,277 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:05,277 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:05,277 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:05,279 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,279 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:05,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:05,281 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:05,281 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:05,281 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:05,281 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:05,281 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,281 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:05,282 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,282 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:05,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,282 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:05,282 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:05,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,283 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:05,286 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:05,286 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,286 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:05,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:56:05,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:05,286 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:05,287 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,287 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:05,287 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:05,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:05,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:05,387 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:05,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:05,391 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:05,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:05,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,393 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:05,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:05,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:05,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:05,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,393 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,394 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:05,394 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:05,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:05,395 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,395 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:05,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,395 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:05,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,396 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:05,396 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,396 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:05,396 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:05,396 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:05,396 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:05,396 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:05,396 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:05,396 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:05,397 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:05,807 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201512288e1f4c4452d34a55aacdbeb12ee66b90 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512288e1f4c4452d34a55aacdbeb12ee66b90 2023-02-12 14:56:05,812 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=21, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/d8234362d5e84368921c3d1eed46293e 2023-02-12 14:56:05,816 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/d8234362d5e84368921c3d1eed46293e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8234362d5e84368921c3d1eed46293e 2023-02-12 14:56:05,820 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8234362d5e84368921c3d1eed46293e, entries=1, sequenceid=21, filesize=4.9 K 2023-02-12 14:56:05,820 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 423ms, sequenceid=21, compaction requested=true 2023-02-12 14:56:05,820 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:05,820 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:05,820 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:05,821 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:05,822 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,822 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:05,822 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,822 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:05,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:05,824 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3f4f95d4[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:05,824 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:05,824 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,825 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,825 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:05,825 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:05,825 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:05,825 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,825 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:05,828 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:05,828 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:05,828 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,828 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:05,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:05,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:05,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:05,830 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:05,830 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:05,830 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:05,830 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:05,830 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,830 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:05,831 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:05,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:05,831 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,831 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:05,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:05,832 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:05,832 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,833 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:05,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:05,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:05,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,838 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:05,838 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:05,838 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,838 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:05,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:05,839 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:05,838 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,939 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:05,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:05,943 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:05,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:05,944 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,944 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:05,944 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:05,944 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:05,944 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:05,945 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,945 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,946 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,946 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:05,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:05,946 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:05,946 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:05,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,947 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,947 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:05,947 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:05,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:05,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:05,947 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:05,948 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:05,948 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:05,948 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:05,948 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:05,948 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:06,134 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:56:06,359 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2015123188086b1e383d432eb131545c173090c8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123188086b1e383d432eb131545c173090c8 2023-02-12 14:56:06,764 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=25, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/dce0af1060f24f94bd1241b359c9ab80 2023-02-12 14:56:06,767 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/dce0af1060f24f94bd1241b359c9ab80 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/dce0af1060f24f94bd1241b359c9ab80 2023-02-12 14:56:06,770 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/dce0af1060f24f94bd1241b359c9ab80, entries=1, sequenceid=25, filesize=4.9 K 2023-02-12 14:56:06,771 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 823ms, sequenceid=25, compaction requested=true 2023-02-12 14:56:06,771 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:06,771 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:06,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:06,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:06,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:06,771 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:06,773 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:06,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:06,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:06,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:06,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:06,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:06,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:06,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ea8a3de[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:06,774 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:06,774 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,776 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,776 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:06,776 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,776 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:06,776 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:06,776 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:06,778 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:06,778 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,778 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:06,778 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:06,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:06,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:06,780 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:06,780 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:06,780 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:06,780 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:06,780 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,780 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:06,781 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,781 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,781 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:06,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:06,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:06,781 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:06,782 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:06,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:06,782 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:06,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,783 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:06,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:06,785 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:06,785 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:06,785 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:06,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:06,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:06,785 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:06,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:06,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:06,885 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:06,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:06,890 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:06,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:06,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,891 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:06,891 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:06,891 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:06,892 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:06,892 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,892 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,893 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,893 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:06,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:06,893 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:06,893 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:06,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,894 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:06,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:06,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:06,894 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:06,894 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:06,894 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:06,895 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:06,895 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:06,895 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:06,895 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:06,895 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:07,309 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601010f684925cfc24c66b946b6be650186bc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601010f684925cfc24c66b946b6be650186bc 2023-02-12 14:56:07,717 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=29, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/257f6b2bc6634f8b8c81ebd960baa35a 2023-02-12 14:56:07,720 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/257f6b2bc6634f8b8c81ebd960baa35a as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/257f6b2bc6634f8b8c81ebd960baa35a 2023-02-12 14:56:07,723 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/257f6b2bc6634f8b8c81ebd960baa35a, entries=1, sequenceid=29, filesize=4.9 K 2023-02-12 14:56:07,724 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 829ms, sequenceid=29, compaction requested=true 2023-02-12 14:56:07,724 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:07,724 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:07,724 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:07,724 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:07,724 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,724 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:07,724 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:07,725 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,725 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:07,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:07,726 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,726 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:07,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:07,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:07,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:07,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:07,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@782c372e[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:07,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:07,727 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,728 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:07,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:07,729 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:07,729 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,729 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:07,731 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:07,731 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,731 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:07,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:07,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:07,731 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:07,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:07,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:07,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:07,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:07,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:07,733 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:07,733 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:07,733 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:07,733 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,733 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:07,735 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,735 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:07,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:07,735 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:07,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,735 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:07,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:07,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,736 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:07,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:07,738 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:07,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:07,738 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:07,738 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:07,738 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:07,739 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:07,739 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:07,739 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:07,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:07,839 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:07,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:07,843 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:07,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:07,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,845 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:07,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:07,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:07,845 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:07,846 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,846 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,847 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:07,847 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:07,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:07,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:07,847 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:07,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,847 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:07,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,848 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,848 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:07,848 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:07,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:07,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:07,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:07,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:07,848 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:07,848 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:07,848 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:07,848 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:08,261 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20160103137678f8ccd14962991c77a374756890 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103137678f8ccd14962991c77a374756890 2023-02-12 14:56:08,665 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=33, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/0be31fcace8043d09e2ededa31c411bd 2023-02-12 14:56:08,669 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/0be31fcace8043d09e2ededa31c411bd as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/0be31fcace8043d09e2ededa31c411bd 2023-02-12 14:56:08,672 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/0be31fcace8043d09e2ededa31c411bd, entries=1, sequenceid=33, filesize=4.9 K 2023-02-12 14:56:08,673 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 825ms, sequenceid=33, compaction requested=true 2023-02-12 14:56:08,673 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:08,673 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:08,673 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:08,673 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:08,673 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,673 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:08,673 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:08,676 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,676 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:08,676 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,676 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:08,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:08,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:08,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:08,677 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2186abed[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:08,677 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:08,677 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,678 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,678 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:08,678 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:08,678 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:08,678 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,679 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:08,680 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,680 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:08,680 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:08,680 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:08,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:08,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:08,682 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:08,682 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:08,682 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:08,682 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,682 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:08,683 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,683 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:08,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,683 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:08,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:08,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:08,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:08,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:08,684 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:08,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,685 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:08,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:08,690 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:08,690 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:08,690 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:08,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:08,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:08,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:08,690 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,791 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:08,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:08,795 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:08,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:08,797 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,797 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:08,797 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:08,797 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:08,797 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:08,797 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,797 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,798 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:08,798 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:08,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:08,799 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:08,799 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:08,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,799 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:08,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,799 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,800 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:08,800 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:08,800 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:08,800 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:08,800 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:08,800 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:08,800 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:08,800 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:08,800 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:08,800 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:09,211 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20150907bb81a1baa71d4d4d935bea2a7ece753e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20150907bb81a1baa71d4d4d935bea2a7ece753e 2023-02-12 14:56:09,616 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=37, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/16bb0e61b28a41baba792776f115a541 2023-02-12 14:56:09,620 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/16bb0e61b28a41baba792776f115a541 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/16bb0e61b28a41baba792776f115a541 2023-02-12 14:56:09,623 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/16bb0e61b28a41baba792776f115a541, entries=1, sequenceid=37, filesize=4.9 K 2023-02-12 14:56:09,624 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 824ms, sequenceid=37, compaction requested=true 2023-02-12 14:56:09,624 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:09,624 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:09,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:09,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:09,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:09,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:09,626 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,626 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:09,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:09,626 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,626 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:09,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:09,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:09,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:09,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:09,627 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@62247b3c[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:09,627 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:09,627 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,628 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:09,628 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:09,628 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,628 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:09,630 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:09,630 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:09,631 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:09,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:09,632 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:09,632 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:09,632 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:09,633 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,633 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:09,634 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,634 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,634 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:09,634 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:09,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,636 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:09,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:09,638 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:09,638 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,638 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy because node does not exist (not an error) 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,638 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:09,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:09,739 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:09,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:09,743 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:09,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMinorCompactionWithMonthlyPolicy was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:09,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,745 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:09,745 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:09,745 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' 2023-02-12 14:56:09,745 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMinorCompactionWithMonthlyPolicy', kicking off acquire phase on members. 2023-02-12 14:56:09,745 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,745 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,746 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,746 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:09,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:09,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:09,747 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:09,747 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:09,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,747 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:09,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,748 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,748 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMinorCompactionWithMonthlyPolicy is still present, removing 2023-02-12 14:56:09,748 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:09,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMinorCompactionWithMonthlyPolicy' with timeout 60000ms 2023-02-12 14:56:09,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:09,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMinorCompactionWithMonthlyPolicy' starting 'acquire' stage 2023-02-12 14:56:09,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-12 14:56:09,748 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:09,748 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:09,748 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. started... 2023-02-12 14:56:09,749 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 98f05e4653d1b15554e6b39d5c185fff 1/1 column families, dataSize=73 B heapSize=408 B 2023-02-12 14:56:10,159 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151120c22c5a35559444e6ac4d1ee69a9ff2b6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120c22c5a35559444e6ac4d1ee69a9ff2b6 2023-02-12 14:56:10,564 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=41, memsize=73, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/773e5075e0ce4b83b88e2baa83c1f61b 2023-02-12 14:56:10,568 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/773e5075e0ce4b83b88e2baa83c1f61b as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/773e5075e0ce4b83b88e2baa83c1f61b 2023-02-12 14:56:10,572 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/773e5075e0ce4b83b88e2baa83c1f61b, entries=1, sequenceid=41, filesize=4.9 K 2023-02-12 14:56:10,572 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~73 B/73, heapSize ~392 B/392, currentSize=0 B/0 for 98f05e4653d1b15554e6b39d5c185fff in 823ms, sequenceid=41, compaction requested=true 2023-02-12 14:56:10,573 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:10,573 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:10,573 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-12 14:56:10,573 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-12 14:56:10,573 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,573 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally acquired 2023-02-12 14:56:10,573 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMinorCompactionWithMonthlyPolicy) in zk 2023-02-12 14:56:10,575 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,575 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:10,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:10,576 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,576 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMinorCompactionWithMonthlyPolicy' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:10,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:10,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:10,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:10,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMinorCompactionWithMonthlyPolicy' on coordinator 2023-02-12 14:56:10,577 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19e992f5[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:10,577 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMinorCompactionWithMonthlyPolicy' starting 'in-barrier' execution. 2023-02-12 14:56:10,577 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,578 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMinorCompactionWithMonthlyPolicy' received 'reached' from coordinator. 2023-02-12 14:56:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMinorCompactionWithMonthlyPolicy' locally completed 2023-02-12 14:56:10,578 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMinorCompactionWithMonthlyPolicy' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:10,578 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,578 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:10,579 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,579 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMinorCompactionWithMonthlyPolicy' has notified controller of completion 2023-02-12 14:56:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:10,579 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:10,580 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMinorCompactionWithMonthlyPolicy' completed. 2023-02-12 14:56:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:10,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMinorCompactionWithMonthlyPolicy' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:10,582 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMinorCompactionWithMonthlyPolicy', counting down latch. Waiting for 0 more 2023-02-12 14:56:10,582 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMinorCompactionWithMonthlyPolicy' execution completed 2023-02-12 14:56:10,582 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:10,582 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:10,582 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,582 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMinorCompactionWithMonthlyPolicyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:10,583 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,583 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:10,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:10,583 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:10,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:10,584 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,585 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,588 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMinorCompactionWithMonthlyPolicy' 2023-02-12 14:56:10,588 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:10,588 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:10,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:10,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMinorCompactionWithMonthlyPolicy'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:10,588 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:10,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:10,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:10,589 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,589 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,589 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,689 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:10,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:10,690 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', MOB_THRESHOLD => '10', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536', MOB_COMPACT_PARTITION_POLICY => 'MONTHLY'}] of the table testMinorCompactionWithMonthlyPolicy 2023-02-12 14:56:10,692 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:56:10,692 DEBUG [PEWorker-1] locking.LockProcedure(309): LOCKED pid=115, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:56:10,693 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:56:10,693 INFO [PEWorker-1] procedure2.TimeoutExecutorThread(81): ADDED pid=115, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214370693 2023-02-12 14:56:10,694 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: false 2023-02-12 14:56:10,695 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 0 del files, 10 selected files, and 0 irrelevant files table 'testMinorCompactionWithMonthlyPolicy' and column 'family1' 2023-02-12 14:56:10,695 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 0 del files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:10,695 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x2dd1abf2 to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:56:10,699 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@49e22372, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:56:10,699 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:11,522 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201601038b8d16615f164cd398664303f8612134 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601038b8d16615f164cd398664303f8612134 2023-02-12 14:56:11,523 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:11,535 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:56:11,537 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:53052, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:56:11,543 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:56:11,544 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34206, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:56:11,553 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/12f381706d53430698bc37317bda8440 first=Optional[r5] last=Optional[r6] 2023-02-12 14:56:11,553 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/12f381706d53430698bc37317bda8440}] 2023-02-12 14:56:11,558 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/12f381706d53430698bc37317bda8440 for inclusion in 98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:11,559 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r5 last=r6 2023-02-12 14:56:11,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:56:11,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:11,560 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101/family1/12f381706d53430698bc37317bda8440 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3ofs2c1i2fr834hful0gna5o3rg29h0lpfg33a25ipa6sk5662ai0ig4g90i41gu/family1/12f381706d53430698bc37317bda8440 2023-02-12 14:56:11,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3ofs2c1i2fr834hful0gna5o3rg29h0lpfg33a25ipa6sk5662ai0ig4g90i41gu/family1/12f381706d53430698bc37317bda8440 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_ 2023-02-12 14:56:11,563 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3ofs2c1i2fr834hful0gna5o3rg29h0lpfg33a25ipa6sk5662ai0ig4g90i41gu/family1/12f381706d53430698bc37317bda8440 into 98f05e4653d1b15554e6b39d5c185fff/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_ - updating store file list. 2023-02-12 14:56:11,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_ into 98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:11,565 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3ofs2c1i2fr834hful0gna5o3rg29h0lpfg33a25ipa6sk5662ai0ig4g90i41gu/family1/12f381706d53430698bc37317bda8440 into 98f05e4653d1b15554e6b39d5c185fff/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_) 2023-02-12 14:56:11,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__3ofs2c1i2fr834hful0gna5o3rg29h0lpfg33a25ipa6sk5662ai0ig4g90i41gu/family1/12f381706d53430698bc37317bda8440 2023-02-12 14:56:11,566 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 11 store files, 0 compacting, 11 eligible, 16 blocking 2023-02-12 14:56:11,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 10 files of size 50903 starting at candidate #1 after considering 44 permutations with 44 in ratio 2023-02-12 14:56:11,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(1912): 98f05e4653d1b15554e6b39d5c185fff/family1 is initiating minor compaction 2023-02-12 14:56:11,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@34aa879f; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:56:11,568 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 98f05e4653d1b15554e6b39d5c185fff/family1 in testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:11,569 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e917cb8429bf49d7915b157e57e91070, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e74ff128405340f892d6d954a99e6325, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/5a0af93be5284cb782a470201c64e51b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8234362d5e84368921c3d1eed46293e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/dce0af1060f24f94bd1241b359c9ab80, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/257f6b2bc6634f8b8c81ebd960baa35a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/0be31fcace8043d09e2ededa31c411bd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/16bb0e61b28a41baba792776f115a541, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/773e5075e0ce4b83b88e2baa83c1f61b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp, totalSize=49.7 K 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting e917cb8429bf49d7915b157e57e91070, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=9 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting e74ff128405340f892d6d954a99e6325, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=13 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 5a0af93be5284cb782a470201c64e51b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=17 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting d8234362d5e84368921c3d1eed46293e, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=21 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting dce0af1060f24f94bd1241b359c9ab80, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=25 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 257f6b2bc6634f8b8c81ebd960baa35a, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=29 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 0be31fcace8043d09e2ededa31c411bd, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=33 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 16bb0e61b28a41baba792776f115a541, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=37 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 773e5075e0ce4b83b88e2baa83c1f61b, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=41 2023-02-12 14:56:11,569 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 00c95ecb598a434e8d776a04e1c51932_SeqId_44_, keycount=2, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=45 2023-02-12 14:56:11,570 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20160101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:11,571 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:11,572 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601010f684925cfc24c66b946b6be650186bc to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201601010f684925cfc24c66b946b6be650186bc 2023-02-12 14:56:11,573 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103137678f8ccd14962991c77a374756890 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20160103137678f8ccd14962991c77a374756890 2023-02-12 14:56:11,573 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 2 expired mob files are deleted 2023-02-12 14:56:11,573 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20160101' 2023-02-12 14:56:11,574 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 2 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20160101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:11,574 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:11,602 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 98f05e4653d1b15554e6b39d5c185fff#family1#compaction#165 average throughput is 0.68 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:12,034 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20151231d2c2fc75163b4b05b9c4bf09cf5e2d51 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151231d2c2fc75163b4b05b9c4bf09cf5e2d51 2023-02-12 14:56:12,034 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:12,058 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ee74d8e89613466da2f67657457ae00f first=Optional[r1] last=Optional[r4] 2023-02-12 14:56:12,058 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ee74d8e89613466da2f67657457ae00f}] 2023-02-12 14:56:12,062 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ee74d8e89613466da2f67657457ae00f for inclusion in 98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:12,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r1 last=r4 2023-02-12 14:56:12,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:56:12,064 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:12,065 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201/family1/ee74d8e89613466da2f67657457ae00f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__dte7cejk7noevs31pqk8honi9hg7o8j7eleekj71js76b3ajdv2gm99djqqcdjk2/family1/ee74d8e89613466da2f67657457ae00f 2023-02-12 14:56:12,066 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__dte7cejk7noevs31pqk8honi9hg7o8j7eleekj71js76b3ajdv2gm99djqqcdjk2/family1/ee74d8e89613466da2f67657457ae00f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_ 2023-02-12 14:56:12,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__dte7cejk7noevs31pqk8honi9hg7o8j7eleekj71js76b3ajdv2gm99djqqcdjk2/family1/ee74d8e89613466da2f67657457ae00f into 98f05e4653d1b15554e6b39d5c185fff/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_ - updating store file list. 2023-02-12 14:56:12,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_ into 98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:12,069 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__dte7cejk7noevs31pqk8honi9hg7o8j7eleekj71js76b3ajdv2gm99djqqcdjk2/family1/ee74d8e89613466da2f67657457ae00f into 98f05e4653d1b15554e6b39d5c185fff/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_) 2023-02-12 14:56:12,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__dte7cejk7noevs31pqk8honi9hg7o8j7eleekj71js76b3ajdv2gm99djqqcdjk2/family1/ee74d8e89613466da2f67657457ae00f 2023-02-12 14:56:12,071 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151201 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:12,072 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:12,073 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512014d34e344d0594caf818427381041d244 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512014d34e344d0594caf818427381041d244 2023-02-12 14:56:12,074 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205de31463a6c1643fb8e0d1e69348c8922 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151205de31463a6c1643fb8e0d1e69348c8922 2023-02-12 14:56:12,075 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512288e1f4c4452d34a55aacdbeb12ee66b90 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201512288e1f4c4452d34a55aacdbeb12ee66b90 2023-02-12 14:56:12,076 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123188086b1e383d432eb131545c173090c8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e2015123188086b1e383d432eb131545c173090c8 2023-02-12 14:56:12,076 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 4 expired mob files are deleted 2023-02-12 14:56:12,076 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151201' 2023-02-12 14:56:12,076 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151201 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:12,076 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:12,428 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/d8180d96b1374cac9f8aeb7d6ecd97fb as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8180d96b1374cac9f8aeb7d6ecd97fb 2023-02-12 14:56:12,432 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 10 file(s) in 98f05e4653d1b15554e6b39d5c185fff/family1 of 98f05e4653d1b15554e6b39d5c185fff into d8180d96b1374cac9f8aeb7d6ecd97fb(size=6.6 K), total size for store is 178.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:56:12,433 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:12,433 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff., storeName=family1, fileCount=10, fileSize=49.7 K (5.0 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 4.9 K, 5.2 K), priority=2, time=1676213771568; duration=0sec 2023-02-12 14:56:12,433 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:56:12,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e917cb8429bf49d7915b157e57e91070, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e74ff128405340f892d6d954a99e6325, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/5a0af93be5284cb782a470201c64e51b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8234362d5e84368921c3d1eed46293e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/dce0af1060f24f94bd1241b359c9ab80, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/257f6b2bc6634f8b8c81ebd960baa35a, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/0be31fcace8043d09e2ededa31c411bd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/16bb0e61b28a41baba792776f115a541, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/773e5075e0ce4b83b88e2baa83c1f61b, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_] to archive 2023-02-12 14:56:12,472 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:12,476 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e917cb8429bf49d7915b157e57e91070 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e917cb8429bf49d7915b157e57e91070 2023-02-12 14:56:12,477 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e74ff128405340f892d6d954a99e6325 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e74ff128405340f892d6d954a99e6325 2023-02-12 14:56:12,478 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/5a0af93be5284cb782a470201c64e51b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/5a0af93be5284cb782a470201c64e51b 2023-02-12 14:56:12,479 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8234362d5e84368921c3d1eed46293e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8234362d5e84368921c3d1eed46293e 2023-02-12 14:56:12,480 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/dce0af1060f24f94bd1241b359c9ab80 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/dce0af1060f24f94bd1241b359c9ab80 2023-02-12 14:56:12,481 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/257f6b2bc6634f8b8c81ebd960baa35a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/257f6b2bc6634f8b8c81ebd960baa35a 2023-02-12 14:56:12,482 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/0be31fcace8043d09e2ededa31c411bd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/0be31fcace8043d09e2ededa31c411bd 2023-02-12 14:56:12,484 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/16bb0e61b28a41baba792776f115a541 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/16bb0e61b28a41baba792776f115a541 2023-02-12 14:56:12,485 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/773e5075e0ce4b83b88e2baa83c1f61b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/773e5075e0ce4b83b88e2baa83c1f61b 2023-02-12 14:56:12,486 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/00c95ecb598a434e8d776a04e1c51932_SeqId_44_ 2023-02-12 14:56:12,908 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e201511302fcd4dbe1be1475791867bcd2d403b7b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e201511302fcd4dbe1be1475791867bcd2d403b7b 2023-02-12 14:56:12,909 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:12,931 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4aa4f7161be4861be580d87006cdad3 first=Optional[r00] last=Optional[r8] 2023-02-12 14:56:12,931 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4aa4f7161be4861be580d87006cdad3}] 2023-02-12 14:56:12,934 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4aa4f7161be4861be580d87006cdad3 for inclusion in 98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:12,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=r00 last=r8 2023-02-12 14:56:12,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last= 2023-02-12 14:56:12,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:12,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101/family1/e4aa4f7161be4861be580d87006cdad3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ah58r8q666lmpc90e97k2i98g9c26mu8o7b6dfihc79ck8mpfernfm3ultlokt9d/family1/e4aa4f7161be4861be580d87006cdad3 2023-02-12 14:56:12,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ah58r8q666lmpc90e97k2i98g9c26mu8o7b6dfihc79ck8mpfernfm3ultlokt9d/family1/e4aa4f7161be4861be580d87006cdad3 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_ 2023-02-12 14:56:12,942 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ah58r8q666lmpc90e97k2i98g9c26mu8o7b6dfihc79ck8mpfernfm3ultlokt9d/family1/e4aa4f7161be4861be580d87006cdad3 into 98f05e4653d1b15554e6b39d5c185fff/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_ - updating store file list. 2023-02-12 14:56:12,945 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_ into 98f05e4653d1b15554e6b39d5c185fff/family1 2023-02-12 14:56:12,945 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ah58r8q666lmpc90e97k2i98g9c26mu8o7b6dfihc79ck8mpfernfm3ultlokt9d/family1/e4aa4f7161be4861be580d87006cdad3 into 98f05e4653d1b15554e6b39d5c185fff/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_) 2023-02-12 14:56:12,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMinorCompactionWithMonthlyPolicy__ah58r8q666lmpc90e97k2i98g9c26mu8o7b6dfihc79ck8mpfernfm3ultlokt9d/family1/e4aa4f7161be4861be580d87006cdad3 2023-02-12 14:56:12,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-12 14:56:12,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 4 files of size 352581 starting at candidate #0 after considering 3 permutations with 3 in ratio 2023-02-12 14:56:12,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(1912): 98f05e4653d1b15554e6b39d5c185fff/family1 is initiating minor compaction (all files) 2023-02-12 14:56:12,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@a4a3433; Because: bulkload hfiles request compaction; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:56:12,947 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 98f05e4653d1b15554e6b39d5c185fff/family1 in testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:12,947 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/eb69b5ec7e2140f081f7b84f58ba40ab, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8180d96b1374cac9f8aeb7d6ecd97fb, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp, totalSize=344.3 K 2023-02-12 14:56:12,948 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting eb69b5ec7e2140f081f7b84f58ba40ab, keycount=1000, bloomtype=ROW, size=166.0 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1448878840591 2023-02-12 14:56:12,948 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting d8180d96b1374cac9f8aeb7d6ecd97fb, keycount=9, bloomtype=ROW, size=6.6 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1441621240591 2023-02-12 14:56:12,948 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting 50e993ba9602484295a2954fb90ccaef_SeqId_46_, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=47, earliestPutTs=1448965240591 2023-02-12 14:56:12,949 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] compactions.Compactor(207): Compacting e77a82cbfad143baa1e660ed88cc9920_SeqId_49_, keycount=1001, bloomtype=ROW, size=166.2 K, encoding=NONE, compression=NONE, seqNum=50, earliestPutTs=1448014840591 2023-02-12 14:56:12,949 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20151101 table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:12,950 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:12,951 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120c22c5a35559444e6ac4d1ee69a9ff2b6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151120c22c5a35559444e6ac4d1ee69a9ff2b6 2023-02-12 14:56:12,952 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b42e87e5a55b45b9ba6e857bd51ee3da to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151128b42e87e5a55b45b9ba6e857bd51ee3da 2023-02-12 14:56:12,953 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130edd1661ed29e4ea0a349ea3f6f1687c4 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/a99017e1e6e4950136225669515fd1e8/family1/d41d8cd98f00b204e9800998ecf8427e20151130edd1661ed29e4ea0a349ea3f6f1687c4 2023-02-12 14:56:12,953 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMinorCompactionWithMonthlyPolicy 3 expired mob files are deleted 2023-02-12 14:56:12,953 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMinorCompactionWithMonthlyPolicy/d41d8cd98f00b204e9800998ecf8427e20151101' 2023-02-12 14:56:12,954 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 3 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20151101 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:12,954 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:12,954 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 1 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20150901 for table='testMinorCompactionWithMonthlyPolicy' and column='family1' 2023-02-12 14:56:12,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 4 mob files. table='testMinorCompactionWithMonthlyPolicy' column='family1' 2023-02-12 14:56:12,954 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMinorCompactionWithMonthlyPolicy', column='family1', allFiles=false, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:56:12,957 DEBUG [PEWorker-3] locking.LockProcedure(242): UNLOCKED pid=115, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE 2023-02-12 14:56:12,958 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=115, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMinorCompactionWithMonthlyPolicy.mobLock, type=EXCLUSIVE in 2.2660 sec 2023-02-12 14:56:12,965 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] throttle.PressureAwareThroughputController(145): 98f05e4653d1b15554e6b39d5c185fff#family1#compaction#166 average throughput is 25.93 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:13,083 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMinorCompactionWithMonthlyPolicy Thread=253 (was 251) - Thread LEAK? -, OpenFileDescriptor=644 (was 631) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=176 (was 198), ProcessCount=168 (was 168), AvailableMemoryMB=3979 (was 3828) - AvailableMemoryMB LEAK? - 2023-02-12 14:56:13,098 INFO [Listener at localhost/37945] hbase.ResourceChecker(147): before: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=253, OpenFileDescriptor=644, MaxFileDescriptor=60000, SystemLoadAverage=176, ProcessCount=168, AvailableMemoryMB=3978 2023-02-12 14:56:13,119 DEBUG [Listener at localhost/37945] crypto.Encryption(565): Installed org.apache.hadoop.hbase.io.crypto.KeyProviderForTesting into key provider cache 2023-02-12 14:56:13,172 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.HMaster$4(2102): Client=jenkins//172.31.14.131 create 'testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} 2023-02-12 14:56:13,173 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure2.ProcedureExecutor(1029): Stored pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testMajorCompactionFromAdmin 2023-02-12 14:56:13,175 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_PRE_OPERATION 2023-02-12 14:56:13,175 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(697): Client=jenkins//172.31.14.131 procedure request for creating table: namespace: "default" qualifier: "testMajorCompactionFromAdmin" procId is: 116 2023-02-12 14:56:13,175 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-12 14:56:13,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-12 14:56:13,177 DEBUG [HFileArchiver-20] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:13,177 DEBUG [HFileArchiver-21] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:13,177 DEBUG [HFileArchiver-22] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:13,177 DEBUG [HFileArchiver-20] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964 empty. 2023-02-12 14:56:13,178 DEBUG [HFileArchiver-22] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9 empty. 2023-02-12 14:56:13,178 DEBUG [HFileArchiver-21] backup.HFileArchiver(153): Directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2 empty. 2023-02-12 14:56:13,178 DEBUG [HFileArchiver-20] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:13,178 DEBUG [HFileArchiver-21] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:13,178 DEBUG [HFileArchiver-22] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:13,178 DEBUG [PEWorker-2] procedure.DeleteTableProcedure(328): Archived testMajorCompactionFromAdmin regions 2023-02-12 14:56:13,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-12 14:56:13,402 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/.tmp/family1/23a09411723d4edb81d222d2e6b26e77 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/23a09411723d4edb81d222d2e6b26e77 2023-02-12 14:56:13,406 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 98f05e4653d1b15554e6b39d5c185fff/family1 of 98f05e4653d1b15554e6b39d5c185fff into 23a09411723d4edb81d222d2e6b26e77(size=167.4 K), total size for store is 167.4 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:56:13,406 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 98f05e4653d1b15554e6b39d5c185fff: 2023-02-12 14:56:13,406 INFO [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff., storeName=family1, fileCount=4, fileSize=344.3 K (166.0 K, 6.6 K, 5.5 K, 166.2 K), priority=2, time=1676213772947; duration=0sec 2023-02-12 14:56:13,407 DEBUG [RS:0;jenkins-hbase4:45319-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-12 14:56:13,470 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/eb69b5ec7e2140f081f7b84f58ba40ab, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8180d96b1374cac9f8aeb7d6ecd97fb, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_] to archive 2023-02-12 14:56:13,471 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:13,472 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/eb69b5ec7e2140f081f7b84f58ba40ab to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/eb69b5ec7e2140f081f7b84f58ba40ab 2023-02-12 14:56:13,474 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8180d96b1374cac9f8aeb7d6ecd97fb to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/d8180d96b1374cac9f8aeb7d6ecd97fb 2023-02-12 14:56:13,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/50e993ba9602484295a2954fb90ccaef_SeqId_46_ 2023-02-12 14:56:13,475 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_ to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/family1/e77a82cbfad143baa1e660ed88cc9920_SeqId_49_ 2023-02-12 14:56:13,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-12 14:56:13,591 DEBUG [PEWorker-2] util.FSTableDescriptors(570): Wrote into hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp/data/default/testMajorCompactionFromAdmin/.tabledesc/.tableinfo.0000000001 2023-02-12 14:56:13,592 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(7671): creating {ENCODED => 18a72c39300757827cdf49fe0a82a4c9, NAME => 'testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9.', STARTKEY => '2', ENDKEY => ''}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:56:13,592 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(7671): creating {ENCODED => a9971a4f82394dd4822a56ba58fc7ed2, NAME => 'testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2.', STARTKEY => '1', ENDKEY => '2'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:56:13,592 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(7671): creating {ENCODED => 9131b0948a7ca3b4830f067f6f120964, NAME => 'testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964.', STARTKEY => '', ENDKEY => '1'}, tableDescriptor='testMajorCompactionFromAdmin', {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}, regionDir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/.tmp 2023-02-12 14:56:13,599 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:13,600 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1603): Closing 18a72c39300757827cdf49fe0a82a4c9, disabling compactions & flushes 2023-02-12 14:56:13,600 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:13,600 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:13,600 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. after waiting 0 ms 2023-02-12 14:56:13,600 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:13,600 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:13,600 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-2] regionserver.HRegion(1557): Region close journal for 18a72c39300757827cdf49fe0a82a4c9: Waiting for close lock at 1676213773600Disabling compacts and flushes for region at 1676213773600Disabling writes for close at 1676213773600Writing region close event to WAL at 1676213773600Closed at 1676213773600 2023-02-12 14:56:13,778 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-12 14:56:14,000 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:14,000 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:14,000 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1603): Closing 9131b0948a7ca3b4830f067f6f120964, disabling compactions & flushes 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1603): Closing a9971a4f82394dd4822a56ba58fc7ed2, disabling compactions & flushes 2023-02-12 14:56:14,001 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:14,001 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. after waiting 0 ms 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:14,001 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. after waiting 0 ms 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-0] regionserver.HRegion(1557): Region close journal for 9131b0948a7ca3b4830f067f6f120964: Waiting for close lock at 1676213774000Disabling compacts and flushes for region at 1676213774000Disabling writes for close at 1676213774001 (+1 ms)Writing region close event to WAL at 1676213774001Closed at 1676213774001 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:14,001 INFO [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:14,001 DEBUG [RegionOpenAndInit-testMajorCompactionFromAdmin-pool-1] regionserver.HRegion(1557): Region close journal for a9971a4f82394dd4822a56ba58fc7ed2: Waiting for close lock at 1676213774001Disabling compacts and flushes for region at 1676213774001Disabling writes for close at 1676213774001Writing region close event to WAL at 1676213774001Closed at 1676213774001 2023-02-12 14:56:14,003 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ADD_TO_META 2023-02-12 14:56:14,004 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213774004"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213774004"}]},"ts":"1676213774004"} 2023-02-12 14:56:14,004 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213774004"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213774004"}]},"ts":"1676213774004"} 2023-02-12 14:56:14,004 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676213774004"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676213774004"}]},"ts":"1676213774004"} 2023-02-12 14:56:14,006 INFO [PEWorker-2] hbase.MetaTableAccessor(1496): Added 3 regions to meta. 2023-02-12 14:56:14,006 INFO [PEWorker-2] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-12 14:56:14,007 DEBUG [PEWorker-2] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213774007"}]},"ts":"1676213774007"} 2023-02-12 14:56:14,008 INFO [PEWorker-2] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLING in hbase:meta 2023-02-12 14:56:14,010 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9131b0948a7ca3b4830f067f6f120964, ASSIGN}, {pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a9971a4f82394dd4822a56ba58fc7ed2, ASSIGN}, {pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18a72c39300757827cdf49fe0a82a4c9, ASSIGN}] 2023-02-12 14:56:14,012 INFO [PEWorker-5] procedure.MasterProcedureScheduler(727): Took xlock for pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18a72c39300757827cdf49fe0a82a4c9, ASSIGN 2023-02-12 14:56:14,012 INFO [PEWorker-4] procedure.MasterProcedureScheduler(727): Took xlock for pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a9971a4f82394dd4822a56ba58fc7ed2, ASSIGN 2023-02-12 14:56:14,012 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9131b0948a7ca3b4830f067f6f120964, ASSIGN 2023-02-12 14:56:14,012 INFO [PEWorker-5] assignment.TransitRegionStateProcedure(193): Starting pid=119, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18a72c39300757827cdf49fe0a82a4c9, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:56:14,012 INFO [PEWorker-4] assignment.TransitRegionStateProcedure(193): Starting pid=118, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a9971a4f82394dd4822a56ba58fc7ed2, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:56:14,012 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=117, ppid=116, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9131b0948a7ca3b4830f067f6f120964, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,45319,1676213617001; forceNewPlan=false, retain=false 2023-02-12 14:56:14,164 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=a9971a4f82394dd4822a56ba58fc7ed2, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:14,164 INFO [PEWorker-3] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=18a72c39300757827cdf49fe0a82a4c9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:14,164 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=9131b0948a7ca3b4830f067f6f120964, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:14,164 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676213774164"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213774164"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213774164"}]},"ts":"1676213774164"} 2023-02-12 14:56:14,164 DEBUG [PEWorker-3] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213774164"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213774164"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213774164"}]},"ts":"1676213774164"} 2023-02-12 14:56:14,164 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213774164"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1676213774164"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676213774164"}]},"ts":"1676213774164"} 2023-02-12 14:56:14,165 INFO [PEWorker-2] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=118, state=RUNNABLE; OpenRegionProcedure a9971a4f82394dd4822a56ba58fc7ed2, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:56:14,166 INFO [PEWorker-5] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=121, ppid=117, state=RUNNABLE; OpenRegionProcedure 9131b0948a7ca3b4830f067f6f120964, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:56:14,167 INFO [PEWorker-3] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=119, state=RUNNABLE; OpenRegionProcedure 18a72c39300757827cdf49fe0a82a4c9, server=jenkins-hbase4.apache.org,45319,1676213617001}] 2023-02-12 14:56:14,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-12 14:56:14,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:14,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a9971a4f82394dd4822a56ba58fc7ed2, NAME => 'testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2.', STARTKEY => '1', ENDKEY => '2'} 2023-02-12 14:56:14,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:14,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,341 INFO [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,342 DEBUG [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1 2023-02-12 14:56:14,343 DEBUG [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1 2023-02-12 14:56:14,343 INFO [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9971a4f82394dd4822a56ba58fc7ed2 columnFamilyName family1 2023-02-12 14:56:14,344 INFO [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] regionserver.HStore(310): Store=a9971a4f82394dd4822a56ba58fc7ed2/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:56:14,344 INFO [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,345 DEBUG [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2 2023-02-12 14:56:14,345 DEBUG [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2 2023-02-12 14:56:14,345 INFO [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a9971a4f82394dd4822a56ba58fc7ed2 columnFamilyName family2 2023-02-12 14:56:14,346 INFO [StoreOpener-a9971a4f82394dd4822a56ba58fc7ed2-1] regionserver.HStore(310): Store=a9971a4f82394dd4822a56ba58fc7ed2/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:56:14,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,347 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:56:14,349 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:14,351 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:56:14,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened a9971a4f82394dd4822a56ba58fc7ed2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68447877, jitterRate=0.019952848553657532}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:56:14,352 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for a9971a4f82394dd4822a56ba58fc7ed2: Running coprocessor pre-open hook at 1676213774320Writing region info on filesystem at 1676213774320Initializing all the Stores at 1676213774321 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213774321Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213774321Cleaning up temporary data from old regions at 1676213774348 (+27 ms)Cleaning up detritus from prior splits at 1676213774348Running coprocessor post-open hooks at 1676213774352 (+4 ms)Region opened successfully at 1676213774352 2023-02-12 14:56:14,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2., pid=120, masterSystemTime=1676213774317 2023-02-12 14:56:14,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:14,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:14,354 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:14,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 9131b0948a7ca3b4830f067f6f120964, NAME => 'testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964.', STARTKEY => '', ENDKEY => '1'} 2023-02-12 14:56:14,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,354 INFO [PEWorker-5] assignment.RegionStateStore(219): pid=118 updating hbase:meta row=a9971a4f82394dd4822a56ba58fc7ed2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:14,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:14,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,354 DEBUG [PEWorker-5] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676213774354"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213774354"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213774354"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213774354"}]},"ts":"1676213774354"} 2023-02-12 14:56:14,354 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,357 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=118 2023-02-12 14:56:14,357 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=118, state=SUCCESS; OpenRegionProcedure a9971a4f82394dd4822a56ba58fc7ed2, server=jenkins-hbase4.apache.org,45319,1676213617001 in 190 msec 2023-02-12 14:56:14,357 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=a9971a4f82394dd4822a56ba58fc7ed2, ASSIGN in 347 msec 2023-02-12 14:56:14,373 INFO [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,375 DEBUG [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1 2023-02-12 14:56:14,375 DEBUG [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1 2023-02-12 14:56:14,375 INFO [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9131b0948a7ca3b4830f067f6f120964 columnFamilyName family1 2023-02-12 14:56:14,376 INFO [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] regionserver.HStore(310): Store=9131b0948a7ca3b4830f067f6f120964/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:56:14,376 INFO [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,377 DEBUG [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2 2023-02-12 14:56:14,377 DEBUG [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2 2023-02-12 14:56:14,377 INFO [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9131b0948a7ca3b4830f067f6f120964 columnFamilyName family2 2023-02-12 14:56:14,378 INFO [StoreOpener-9131b0948a7ca3b4830f067f6f120964-1] regionserver.HStore(310): Store=9131b0948a7ca3b4830f067f6f120964/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:56:14,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,379 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:56:14,382 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:14,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:56:14,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 9131b0948a7ca3b4830f067f6f120964; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66947342, jitterRate=-0.002406865358352661}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:56:14,383 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 9131b0948a7ca3b4830f067f6f120964: Running coprocessor pre-open hook at 1676213774354Writing region info on filesystem at 1676213774354Initializing all the Stores at 1676213774355 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213774355Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213774355Cleaning up temporary data from old regions at 1676213774380 (+25 ms)Cleaning up detritus from prior splits at 1676213774380Running coprocessor post-open hooks at 1676213774383 (+3 ms)Region opened successfully at 1676213774383 2023-02-12 14:56:14,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964., pid=121, masterSystemTime=1676213774317 2023-02-12 14:56:14,385 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:14,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:14,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(128): Open testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:14,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 18a72c39300757827cdf49fe0a82a4c9, NAME => 'testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9.', STARTKEY => '2', ENDKEY => ''} 2023-02-12 14:56:14,386 INFO [PEWorker-2] assignment.RegionStateStore(219): pid=117 updating hbase:meta row=9131b0948a7ca3b4830f067f6f120964, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:14,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testMajorCompactionFromAdmin 18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,386 DEBUG [PEWorker-2] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213774386"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213774386"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213774386"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213774386"}]},"ts":"1676213774386"} 2023-02-12 14:56:14,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(865): Instantiated testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-12 14:56:14,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7890): checking encryption for 18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,386 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7893): checking classloading for 18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,388 INFO [PEWorker-2] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=117 2023-02-12 14:56:14,388 INFO [PEWorker-2] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=117, state=SUCCESS; OpenRegionProcedure 9131b0948a7ca3b4830f067f6f120964, server=jenkins-hbase4.apache.org,45319,1676213617001 in 221 msec 2023-02-12 14:56:14,389 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=9131b0948a7ca3b4830f067f6f120964, ASSIGN in 378 msec 2023-02-12 14:56:14,406 INFO [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family1 of region 18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,407 DEBUG [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1 2023-02-12 14:56:14,407 DEBUG [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1 2023-02-12 14:56:14,408 INFO [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18a72c39300757827cdf49fe0a82a4c9 columnFamilyName family1 2023-02-12 14:56:14,408 INFO [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] regionserver.HStore(310): Store=18a72c39300757827cdf49fe0a82a4c9/family1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:56:14,408 INFO [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family family2 of region 18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,409 DEBUG [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2 2023-02-12 14:56:14,409 DEBUG [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2 2023-02-12 14:56:14,410 INFO [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 604800000, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18a72c39300757827cdf49fe0a82a4c9 columnFamilyName family2 2023-02-12 14:56:14,410 INFO [StoreOpener-18a72c39300757827cdf49fe0a82a4c9-1] regionserver.HStore(310): Store=18a72c39300757827cdf49fe0a82a4c9/family2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-12 14:56:14,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testMajorCompactionFromAdmin descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-12 14:56:14,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1054): writing seq id for 18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:14,414 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-12 14:56:14,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1071): Opened 18a72c39300757827cdf49fe0a82a4c9; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60078771, jitterRate=-0.10475654900074005}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-12 14:56:14,415 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(964): Region open journal for 18a72c39300757827cdf49fe0a82a4c9: Running coprocessor pre-open hook at 1676213774386Writing region info on filesystem at 1676213774386Initializing all the Stores at 1676213774387 (+1 ms)Instantiating store for column family {NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213774387Instantiating store for column family {NAME => 'family2', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'} at 1676213774387Cleaning up temporary data from old regions at 1676213774412 (+25 ms)Cleaning up detritus from prior splits at 1676213774412Running coprocessor post-open hooks at 1676213774415 (+3 ms)Region opened successfully at 1676213774415 2023-02-12 14:56:14,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9., pid=122, masterSystemTime=1676213774317 2023-02-12 14:56:14,416 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:14,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(156): Opened testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:14,417 INFO [PEWorker-4] assignment.RegionStateStore(219): pid=119 updating hbase:meta row=18a72c39300757827cdf49fe0a82a4c9, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:14,417 DEBUG [PEWorker-4] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9.","families":{"info":[{"qualifier":"regioninfo","vlen":63,"tag":[],"timestamp":"1676213774416"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1676213774416"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676213774416"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676213774416"}]},"ts":"1676213774416"} 2023-02-12 14:56:14,418 INFO [PEWorker-4] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=119 2023-02-12 14:56:14,419 INFO [PEWorker-4] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=119, state=SUCCESS; OpenRegionProcedure 18a72c39300757827cdf49fe0a82a4c9, server=jenkins-hbase4.apache.org,45319,1676213617001 in 252 msec 2023-02-12 14:56:14,420 INFO [PEWorker-5] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=119, resume processing ppid=116 2023-02-12 14:56:14,420 INFO [PEWorker-5] procedure2.ProcedureExecutor(1410): Finished pid=119, ppid=116, state=SUCCESS; TransitRegionStateProcedure table=testMajorCompactionFromAdmin, region=18a72c39300757827cdf49fe0a82a4c9, ASSIGN in 409 msec 2023-02-12 14:56:14,420 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-12 14:56:14,420 DEBUG [PEWorker-3] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testMajorCompactionFromAdmin","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676213774420"}]},"ts":"1676213774420"} 2023-02-12 14:56:14,421 INFO [PEWorker-3] hbase.MetaTableAccessor(1635): Updated tableName=testMajorCompactionFromAdmin, state=ENABLED in hbase:meta 2023-02-12 14:56:14,423 INFO [PEWorker-3] procedure.CreateTableProcedure(80): pid=116, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testMajorCompactionFromAdmin execute state=CREATE_TABLE_POST_OPERATION 2023-02-12 14:56:14,425 INFO [PEWorker-3] procedure2.ProcedureExecutor(1410): Finished pid=116, state=SUCCESS; CreateTableProcedure table=testMajorCompactionFromAdmin in 1.2510 sec 2023-02-12 14:56:15,280 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1227): Checking to see if procedure is done pid=116 2023-02-12 14:56:15,280 INFO [Listener at localhost/37945] client.HBaseAdmin$TableFuture(3541): Operation: CREATE, Table Name: default:testMajorCompactionFromAdmin, procId: 116 completed 2023-02-12 14:56:15,286 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:15,287 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:15,287 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:15,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:15,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-12 14:56:15,296 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-12 14:56:15,296 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:15,296 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-12 14:56:15,296 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-12 14:56:15,296 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,296 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,299 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:15,299 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:15,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:15,299 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,299 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:15,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:15,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-12 14:56:15,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-12 14:56:15,300 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-12 14:56:15,300 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:15,300 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-12 14:56:15,300 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:15,300 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. started... 2023-02-12 14:56:15,300 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:56:15,300 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:15,301 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9971a4f82394dd4822a56ba58fc7ed2 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:56:15,301 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. started... 2023-02-12 14:56:15,301 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:15,300 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:15,301 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. started... 2023-02-12 14:56:15,302 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18a72c39300757827cdf49fe0a82a4c9 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:56:15,302 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9131b0948a7ca3b4830f067f6f120964 2/2 column families, dataSize=342 B heapSize=1.27 KB 2023-02-12 14:56:15,351 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212ea4c31f842854a6c842c19e50fcf5b1d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212ea4c31f842854a6c842c19e50fcf5b1d 2023-02-12 14:56:15,354 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302123e7c8237cbd74151908117ad92ecb428 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302123e7c8237cbd74151908117ad92ecb428 2023-02-12 14:56:15,355 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302124d552cbb411d4b4a8b6c96b6d8553aba to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302124d552cbb411d4b4a8b6c96b6d8553aba 2023-02-12 14:56:15,366 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/f2b958201294424a8699505910a16c95 2023-02-12 14:56:15,369 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d 2023-02-12 14:56:15,372 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=228, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/a0a03715c3384576be5b4cc537dd8e0c 2023-02-12 14:56:15,407 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212b819522a39c54354b5476c683fb1a502 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230212b819522a39c54354b5476c683fb1a502 2023-02-12 14:56:15,424 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212c510ee9069074e45a22e80e33f0737c7 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230212c510ee9069074e45a22e80e33f0737c7 2023-02-12 14:56:15,818 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/5ea3891f1ac448f689a162ed60ceb544 2023-02-12 14:56:15,821 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212333ed798b24847b5a89d31aaebe96043 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230212333ed798b24847b5a89d31aaebe96043 2023-02-12 14:56:15,822 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d 2023-02-12 14:56:15,826 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/6274e7dd267642a8a7320a22d406ff27 2023-02-12 14:56:15,829 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/a0a03715c3384576be5b4cc537dd8e0c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a0a03715c3384576be5b4cc537dd8e0c 2023-02-12 14:56:15,829 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=5, memsize=114, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/3e41637e3be042cd8a51f56a875b522f 2023-02-12 14:56:15,833 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/f2b958201294424a8699505910a16c95 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f2b958201294424a8699505910a16c95 2023-02-12 14:56:15,856 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d, entries=4, sequenceid=5, filesize=5.5 K 2023-02-12 14:56:15,857 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/5ea3891f1ac448f689a162ed60ceb544 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/5ea3891f1ac448f689a162ed60ceb544 2023-02-12 14:56:15,860 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a0a03715c3384576be5b4cc537dd8e0c, entries=4, sequenceid=5, filesize=5.5 K 2023-02-12 14:56:15,861 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/6274e7dd267642a8a7320a22d406ff27 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/6274e7dd267642a8a7320a22d406ff27 2023-02-12 14:56:15,861 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/5ea3891f1ac448f689a162ed60ceb544, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:56:15,862 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for a9971a4f82394dd4822a56ba58fc7ed2 in 561ms, sequenceid=5, compaction requested=false 2023-02-12 14:56:15,862 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testMajorCompactionFromAdmin' 2023-02-12 14:56:15,862 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:15,863 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:15,863 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:56:15,864 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f2b958201294424a8699505910a16c95, entries=4, sequenceid=5, filesize=5.5 K 2023-02-12 14:56:15,864 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/6274e7dd267642a8a7320a22d406ff27, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:56:15,865 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/3e41637e3be042cd8a51f56a875b522f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/3e41637e3be042cd8a51f56a875b522f 2023-02-12 14:56:15,865 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 9131b0948a7ca3b4830f067f6f120964 in 563ms, sequenceid=5, compaction requested=false 2023-02-12 14:56:15,866 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:15,866 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:15,866 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:56:15,869 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/3e41637e3be042cd8a51f56a875b522f, entries=2, sequenceid=5, filesize=5.1 K 2023-02-12 14:56:15,869 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~342 B/342, heapSize ~1.24 KB/1272, currentSize=0 B/0 for 18a72c39300757827cdf49fe0a82a4c9 in 567ms, sequenceid=5, compaction requested=false 2023-02-12 14:56:15,869 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:15,869 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:15,869 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:56:15,870 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:56:15,870 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,870 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-12 14:56:15,870 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-12 14:56:15,874 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,874 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:15,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:15,874 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,874 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:15,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:15,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:15,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:15,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:15,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-12 14:56:15,875 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e543a1[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:15,875 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-12 14:56:15,875 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,876 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,876 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,877 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-12 14:56:15,877 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-12 14:56:15,877 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:15,877 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,877 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:15,879 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-12 14:56:15,879 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,879 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:15,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:15,879 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-12 14:56:15,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:15,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:15,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:15,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:15,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-12 14:56:15,882 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-12 14:56:15,882 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:15,882 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:15,882 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-12 14:56:15,882 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:15,884 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,884 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:15,884 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:15,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:15,885 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:15,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:15,886 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:15,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:15,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,887 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:15,888 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-12 14:56:15,888 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:15,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:15,888 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-12 14:56:15,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:15,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:15,888 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,988 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:15,989 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:15,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:15,990 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:15,991 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:15,993 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:15,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:15,995 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-12 14:56:15,995 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-12 14:56:15,995 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:15,995 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-12 14:56:15,995 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-12 14:56:15,996 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,996 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,997 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,997 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:15,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:15,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:15,997 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:15,997 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,998 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:15,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:15,998 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-12 14:56:15,998 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-12 14:56:15,998 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-12 14:56:15,998 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-12 14:56:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-12 14:56:15,999 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:15,999 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:56:15,999 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. started... 2023-02-12 14:56:15,999 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:15,999 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. started... 2023-02-12 14:56:15,999 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. started... 2023-02-12 14:56:15,999 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18a72c39300757827cdf49fe0a82a4c9 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:56:15,999 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9971a4f82394dd4822a56ba58fc7ed2 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:56:15,999 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:15,999 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9131b0948a7ca3b4830f067f6f120964 2/2 column families, dataSize=462 B heapSize=1.39 KB 2023-02-12 14:56:16,044 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302120298c9e7962346c9afa3322176122b1f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302120298c9e7962346c9afa3322176122b1f 2023-02-12 14:56:16,058 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/cd1faa0085a041ccbbc77d5cdca99342 2023-02-12 14:56:16,440 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302123cc6f37097454d4d8917120d52a07ecf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302123cc6f37097454d4d8917120d52a07ecf 2023-02-12 14:56:16,445 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021290c59c318a8d4acebc27f718c933b0c8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021290c59c318a8d4acebc27f718c933b0c8 2023-02-12 14:56:16,498 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021250725dcc3d3b4d6cbec3ac68d85be7ed to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b2023021250725dcc3d3b4d6cbec3ac68d85be7ed 2023-02-12 14:56:16,859 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/a6a22c390ac9428aba57a12d959bd2ab 2023-02-12 14:56:16,862 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=308, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/68b0b14982484823ab624417dcfb8d87 2023-02-12 14:56:16,903 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/9061beb742e246b79bda542b0d93c427 2023-02-12 14:56:16,906 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/cd1faa0085a041ccbbc77d5cdca99342 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/cd1faa0085a041ccbbc77d5cdca99342 2023-02-12 14:56:16,929 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/cd1faa0085a041ccbbc77d5cdca99342, entries=4, sequenceid=9, filesize=5.5 K 2023-02-12 14:56:16,930 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/9061beb742e246b79bda542b0d93c427 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/9061beb742e246b79bda542b0d93c427 2023-02-12 14:56:16,933 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/9061beb742e246b79bda542b0d93c427, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:56:16,934 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for a9971a4f82394dd4822a56ba58fc7ed2 in 935ms, sequenceid=9, compaction requested=false 2023-02-12 14:56:16,934 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:16,934 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:16,934 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:56:17,291 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212794615f747be491b8c258ba2c2d5592d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230212794615f747be491b8c258ba2c2d5592d 2023-02-12 14:56:17,304 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021244c05fe4cd7646c28f8153928c7b16e3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c2023021244c05fe4cd7646c28f8153928c7b16e3 2023-02-12 14:56:17,696 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/d9aa061f9cdd495c833bc17098dfe3a8 2023-02-12 14:56:17,699 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/a6a22c390ac9428aba57a12d959bd2ab as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a6a22c390ac9428aba57a12d959bd2ab 2023-02-12 14:56:17,709 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=9, memsize=154, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/2e85281cd4c44099a6bdbf47df6e7ff5 2023-02-12 14:56:17,712 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/68b0b14982484823ab624417dcfb8d87 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/68b0b14982484823ab624417dcfb8d87 2023-02-12 14:56:17,723 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a6a22c390ac9428aba57a12d959bd2ab, entries=4, sequenceid=9, filesize=5.5 K 2023-02-12 14:56:17,724 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/d9aa061f9cdd495c833bc17098dfe3a8 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d9aa061f9cdd495c833bc17098dfe3a8 2023-02-12 14:56:17,727 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d9aa061f9cdd495c833bc17098dfe3a8, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:56:17,727 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 9131b0948a7ca3b4830f067f6f120964 in 1728ms, sequenceid=9, compaction requested=false 2023-02-12 14:56:17,728 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:17,728 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:17,728 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:56:17,746 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/68b0b14982484823ab624417dcfb8d87, entries=4, sequenceid=9, filesize=5.5 K 2023-02-12 14:56:17,747 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/2e85281cd4c44099a6bdbf47df6e7ff5 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/2e85281cd4c44099a6bdbf47df6e7ff5 2023-02-12 14:56:17,750 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/2e85281cd4c44099a6bdbf47df6e7ff5, entries=2, sequenceid=9, filesize=5.1 K 2023-02-12 14:56:17,751 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~462 B/462, heapSize ~1.36 KB/1392, currentSize=0 B/0 for 18a72c39300757827cdf49fe0a82a4c9 in 1752ms, sequenceid=9, compaction requested=false 2023-02-12 14:56:17,751 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:17,751 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:17,751 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:56:17,752 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:56:17,752 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,752 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-12 14:56:17,752 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-12 14:56:17,753 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,753 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:17,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:17,753 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:17,753 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:17,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:17,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:17,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:17,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-12 14:56:17,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@bf63cc[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:17,755 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-12 14:56:17,755 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,756 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-12 14:56:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-12 14:56:17,756 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:17,756 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,756 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:17,757 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-12 14:56:17,757 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,757 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:17,757 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-12 14:56:17,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:17,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:17,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:17,759 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-12 14:56:17,759 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-12 14:56:17,759 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:17,759 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:17,759 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-12 14:56:17,759 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:17,762 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,762 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:17,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,762 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:17,762 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:17,762 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:17,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:17,763 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:17,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:17,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:17,765 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:17,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,765 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:17,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:17,765 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:17,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:17,766 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,766 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:17,766 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:17,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:17,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:17,866 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:17,867 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:17,868 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:17,868 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:17,868 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:17,870 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:17,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:17,872 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-12 14:56:17,872 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-12 14:56:17,872 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:17,872 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-12 14:56:17,872 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-12 14:56:17,872 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,873 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:17,874 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,874 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:17,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:17,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:17,874 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:17,874 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:17,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:17,875 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:17,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:17,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:17,875 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-12 14:56:17,875 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-12 14:56:17,875 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-12 14:56:17,875 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-12 14:56:17,875 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:17,875 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-12 14:56:17,876 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:17,876 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. started... 2023-02-12 14:56:17,876 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9971a4f82394dd4822a56ba58fc7ed2 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:56:17,876 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:17,876 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:56:17,876 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:17,876 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:17,876 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. started... 2023-02-12 14:56:17,876 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. started... 2023-02-12 14:56:17,876 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18a72c39300757827cdf49fe0a82a4c9 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:56:17,876 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9131b0948a7ca3b4830f067f6f120964 2/2 column families, dataSize=582 B heapSize=1.51 KB 2023-02-12 14:56:18,321 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212c2b45f15790643528b12d40ef6b65ce0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230212c2b45f15790643528b12d40ef6b65ce0 2023-02-12 14:56:18,327 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021231a56592fc364779b7a62e0d168f3a8d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021231a56592fc364779b7a62e0d168f3a8d 2023-02-12 14:56:18,336 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302129785bb7af6884ab1a573690f51770179 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302129785bb7af6884ab1a573690f51770179 2023-02-12 14:56:18,337 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/9879983a73e9452f9920f1d46d7373a6 2023-02-12 14:56:18,353 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/f67e2f196f284a2690f96eb444217615 2023-02-12 14:56:18,746 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=388, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/f7a4044b61b84c7881d467a0722adc03 2023-02-12 14:56:18,786 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302127c011148a8134d8abe2dff7c92a37c11 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302127c011148a8134d8abe2dff7c92a37c11 2023-02-12 14:56:18,786 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212c5f5f433fcb447b382d2070d716f757d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230212c5f5f433fcb447b382d2070d716f757d 2023-02-12 14:56:19,186 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212099322d40a25432d9fcc459c3a8ede61 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c20230212099322d40a25432d9fcc459c3a8ede61 2023-02-12 14:56:19,191 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/b5089a432a154c5abbbf3a87d96ad3dd 2023-02-12 14:56:19,192 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/befd5edcbe694722805cf968247c12e9 2023-02-12 14:56:19,195 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/9879983a73e9452f9920f1d46d7373a6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/9879983a73e9452f9920f1d46d7373a6 2023-02-12 14:56:19,196 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/f67e2f196f284a2690f96eb444217615 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/f67e2f196f284a2690f96eb444217615 2023-02-12 14:56:19,218 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/9879983a73e9452f9920f1d46d7373a6, entries=4, sequenceid=13, filesize=5.5 K 2023-02-12 14:56:19,218 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/f67e2f196f284a2690f96eb444217615, entries=4, sequenceid=13, filesize=5.5 K 2023-02-12 14:56:19,219 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/b5089a432a154c5abbbf3a87d96ad3dd as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/b5089a432a154c5abbbf3a87d96ad3dd 2023-02-12 14:56:19,219 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/befd5edcbe694722805cf968247c12e9 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/befd5edcbe694722805cf968247c12e9 2023-02-12 14:56:19,222 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/b5089a432a154c5abbbf3a87d96ad3dd, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:56:19,222 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/befd5edcbe694722805cf968247c12e9, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:56:19,223 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for a9971a4f82394dd4822a56ba58fc7ed2 in 1347ms, sequenceid=13, compaction requested=true 2023-02-12 14:56:19,223 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 9131b0948a7ca3b4830f067f6f120964 in 1347ms, sequenceid=13, compaction requested=true 2023-02-12 14:56:19,223 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:19,223 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:19,223 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:19,223 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:19,223 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:56:19,223 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:56:19,591 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=13, memsize=194, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/71a4039d96ec481abffc241f81792ae8 2023-02-12 14:56:19,595 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/f7a4044b61b84c7881d467a0722adc03 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f7a4044b61b84c7881d467a0722adc03 2023-02-12 14:56:19,617 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f7a4044b61b84c7881d467a0722adc03, entries=4, sequenceid=13, filesize=5.5 K 2023-02-12 14:56:19,618 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/71a4039d96ec481abffc241f81792ae8 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/71a4039d96ec481abffc241f81792ae8 2023-02-12 14:56:19,621 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/71a4039d96ec481abffc241f81792ae8, entries=2, sequenceid=13, filesize=5.1 K 2023-02-12 14:56:19,622 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~582 B/582, heapSize ~1.48 KB/1512, currentSize=0 B/0 for 18a72c39300757827cdf49fe0a82a4c9 in 1746ms, sequenceid=13, compaction requested=true 2023-02-12 14:56:19,622 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:19,622 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:19,622 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:56:19,622 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:56:19,622 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,622 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-12 14:56:19,622 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-12 14:56:19,624 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:19,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,624 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:19,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:19,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-12 14:56:19,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2aedc031[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:19,626 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-12 14:56:19,626 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,630 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,631 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-12 14:56:19,631 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-12 14:56:19,631 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:19,631 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,631 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:19,632 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-12 14:56:19,632 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,632 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:19,632 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-12 14:56:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:19,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:19,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:19,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:19,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:19,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:19,649 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-12 14:56:19,649 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-12 14:56:19,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:19,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:19,649 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-12 14:56:19,649 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:19,650 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:56:19,650 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x44f9d106 to 127.0.0.1:61565 2023-02-12 14:56:19,651 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:19,651 INFO [Finalizer] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:56:19,651 DEBUG [Finalizer] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2dd1abf2 to 127.0.0.1:61565 2023-02-12 14:56:19,651 DEBUG [Finalizer] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:19,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,656 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,656 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:19,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:19,658 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:19,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:19,659 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:19,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:19,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:19,661 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-12 14:56:19,661 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:19,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:19,661 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:19,661 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:19,662 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:19,662 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,762 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:19,762 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:19,763 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:19,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:19,764 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(8801): writing data to region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. with WAL disabled. Data may be lost in the event of a crash. 2023-02-12 14:56:19,766 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:19,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:19,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-12 14:56:19,767 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-12 14:56:19,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:19,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-12 14:56:19,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-12 14:56:19,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,768 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:19,772 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,772 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:19,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:19,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:19,772 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:19,772 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:19,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:19,772 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:19,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:19,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:19,773 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-12 14:56:19,773 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-12 14:56:19,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-12 14:56:19,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:19,773 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-12 14:56:19,773 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:19,773 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:19,774 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:19,774 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. started... 2023-02-12 14:56:19,774 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:56:19,774 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:19,774 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9971a4f82394dd4822a56ba58fc7ed2 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:56:19,774 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. started... 2023-02-12 14:56:19,774 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. started... 2023-02-12 14:56:19,774 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9131b0948a7ca3b4830f067f6f120964 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:56:19,774 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18a72c39300757827cdf49fe0a82a4c9 2/2 column families, dataSize=702 B heapSize=1.63 KB 2023-02-12 14:56:19,820 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021273486115f203438db23a69471c7aaec5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021273486115f203438db23a69471c7aaec5 2023-02-12 14:56:19,872 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-12 14:56:20,219 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212d0fbfd30751d49b8bc25f35012e85474 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212d0fbfd30751d49b8bc25f35012e85474 2023-02-12 14:56:20,229 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021276a93aad046b4596a8fe53da99c7961a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021276a93aad046b4596a8fe53da99c7961a 2023-02-12 14:56:20,234 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/a7b15909520e469cb851c75f827b705e 2023-02-12 14:56:20,234 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/b4e794cc9fb14511b3a00757b7c21512 2023-02-12 14:56:20,281 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302122703e98c38f04f35b0910257a3c332d1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302122703e98c38f04f35b0910257a3c332d1 2023-02-12 14:56:20,285 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/13575bfebf3740f38c3f28b303a7f823 2023-02-12 14:56:20,288 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/b4e794cc9fb14511b3a00757b7c21512 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/b4e794cc9fb14511b3a00757b7c21512 2023-02-12 14:56:20,310 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/b4e794cc9fb14511b3a00757b7c21512, entries=4, sequenceid=17, filesize=5.5 K 2023-02-12 14:56:20,311 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/13575bfebf3740f38c3f28b303a7f823 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/13575bfebf3740f38c3f28b303a7f823 2023-02-12 14:56:20,314 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/13575bfebf3740f38c3f28b303a7f823, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:56:20,315 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 18a72c39300757827cdf49fe0a82a4c9 in 541ms, sequenceid=17, compaction requested=true 2023-02-12 14:56:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:20,315 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:20,315 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:56:20,655 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=468, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/4f47ecb8410243bbb347e6e8c4d75e7f 2023-02-12 14:56:20,672 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b20230212b0e2ed1f09d3443eb4cf37738ce88b83 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b20230212b0e2ed1f09d3443eb4cf37738ce88b83 2023-02-12 14:56:20,677 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/f70b71de0e9a4dd7bb2a08c77991acd6 2023-02-12 14:56:20,681 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/a7b15909520e469cb851c75f827b705e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/a7b15909520e469cb851c75f827b705e 2023-02-12 14:56:20,692 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212fea28117fc67438cbde85b443e9205e6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e20230212fea28117fc67438cbde85b443e9205e6 2023-02-12 14:56:20,714 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/a7b15909520e469cb851c75f827b705e, entries=4, sequenceid=17, filesize=5.5 K 2023-02-12 14:56:20,715 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/f70b71de0e9a4dd7bb2a08c77991acd6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/f70b71de0e9a4dd7bb2a08c77991acd6 2023-02-12 14:56:20,719 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/f70b71de0e9a4dd7bb2a08c77991acd6, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:56:20,719 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for a9971a4f82394dd4822a56ba58fc7ed2 in 945ms, sequenceid=17, compaction requested=true 2023-02-12 14:56:20,720 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:20,720 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:20,720 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:56:21,102 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=17, memsize=234, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/b247a3be051e47938971f6b8e6f611d7 2023-02-12 14:56:21,106 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/4f47ecb8410243bbb347e6e8c4d75e7f as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/4f47ecb8410243bbb347e6e8c4d75e7f 2023-02-12 14:56:21,128 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/4f47ecb8410243bbb347e6e8c4d75e7f, entries=4, sequenceid=17, filesize=5.5 K 2023-02-12 14:56:21,129 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/b247a3be051e47938971f6b8e6f611d7 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/b247a3be051e47938971f6b8e6f611d7 2023-02-12 14:56:21,132 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/b247a3be051e47938971f6b8e6f611d7, entries=2, sequenceid=17, filesize=5.1 K 2023-02-12 14:56:21,133 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~702 B/702, heapSize ~1.59 KB/1632, currentSize=0 B/0 for 9131b0948a7ca3b4830f067f6f120964 in 1359ms, sequenceid=17, compaction requested=true 2023-02-12 14:56:21,133 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:21,133 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:56:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:56:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-12 14:56:21,133 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-12 14:56:21,136 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,136 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:21,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-12 14:56:21,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7c5f702e[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:21,137 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-12 14:56:21,137 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,138 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,138 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-12 14:56:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-12 14:56:21,138 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:21,138 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,139 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-12 14:56:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:21,140 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,140 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-12 14:56:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:21,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:21,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:21,142 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-12 14:56:21,142 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-12 14:56:21,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:21,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:21,142 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-12 14:56:21,142 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:21,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:21,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,148 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:21,149 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:21,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:21,150 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:21,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:21,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,155 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:21,155 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:21,156 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:21,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-12 14:56:21,156 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:21,156 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin because node does not exist (not an error) 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:21,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:21,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,156 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:21,256 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:21,257 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:21,265 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:56:21,272 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:56:21,281 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:56:21,282 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:56:21,369 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-02-12 14:56:21,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(143): Procedure testMajorCompactionFromAdmin was in running list but was completed. Accepting new attempt. 2023-02-12 14:56:21,370 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] procedure.ProcedureCoordinator(165): Submitting procedure testMajorCompactionFromAdmin 2023-02-12 14:56:21,370 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testMajorCompactionFromAdmin' 2023-02-12 14:56:21,370 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:21,370 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testMajorCompactionFromAdmin' starting 'acquire' 2023-02-12 14:56:21,370 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testMajorCompactionFromAdmin', kicking off acquire phase on members. 2023-02-12 14:56:21,371 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,371 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:21,372 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:21,372 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:21,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:21,373 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:21,373 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-12 14:56:21,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:21,373 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:21,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-12 14:56:21,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:21,374 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testMajorCompactionFromAdmin 2023-02-12 14:56:21,375 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testMajorCompactionFromAdmin is still present, removing 2023-02-12 14:56:21,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testMajorCompactionFromAdmin 2023-02-12 14:56:21,375 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testMajorCompactionFromAdmin' with timeout 60000ms 2023-02-12 14:56:21,375 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-12 14:56:21,375 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testMajorCompactionFromAdmin' starting 'acquire' stage 2023-02-12 14:56:21,375 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:21,375 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. started... 2023-02-12 14:56:21,375 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing a9971a4f82394dd4822a56ba58fc7ed2 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:56:21,376 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:21,376 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. started... 2023-02-12 14:56:21,376 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2744): Flushing 18a72c39300757827cdf49fe0a82a4c9 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:56:21,376 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 3 regions 2023-02-12 14:56:21,376 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-12 14:56:21,376 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:21,377 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. started... 2023-02-12 14:56:21,377 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2744): Flushing 9131b0948a7ca3b4830f067f6f120964 2/2 column families, dataSize=138 B heapSize=944 B 2023-02-12 14:56:21,424 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/c68a5cf40fb84f33ade2b7d9644b9eb3 2023-02-12 14:56:21,455 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c68a5cf40fb84f33ade2b7d9644b9eb3 2023-02-12 14:56:21,846 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/ca8297204beb412588538111ca915513 2023-02-12 14:56:21,882 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca8297204beb412588538111ca915513 2023-02-12 14:56:22,240 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=106, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/dc651374f4724cc79b5fe8fd723be01e 2023-02-12 14:56:22,268 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc651374f4724cc79b5fe8fd723be01e 2023-02-12 14:56:22,269 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3 2023-02-12 14:56:22,272 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b5bb1ccce28470abd1e31e0bf9c3cb3 2023-02-12 14:56:22,273 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/c68a5cf40fb84f33ade2b7d9644b9eb3 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/c68a5cf40fb84f33ade2b7d9644b9eb3 2023-02-12 14:56:22,284 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/bd7acbda081945a3a2659acf0135d37c 2023-02-12 14:56:22,288 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd7acbda081945a3a2659acf0135d37c 2023-02-12 14:56:22,289 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/dc651374f4724cc79b5fe8fd723be01e as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/dc651374f4724cc79b5fe8fd723be01e 2023-02-12 14:56:22,297 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] mob.DefaultMobStoreFlusher(134): Mob store is flushed, sequenceid=23, memsize=32, hasBloomFilter=true, into tmp file hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/d82cb284ee204549bf195c2fe9a7a8d0 2023-02-12 14:56:22,297 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c68a5cf40fb84f33ade2b7d9644b9eb3 2023-02-12 14:56:22,297 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/c68a5cf40fb84f33ade2b7d9644b9eb3, entries=3, sequenceid=23, filesize=5.2 K 2023-02-12 14:56:22,297 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3 2023-02-12 14:56:22,301 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2b5bb1ccce28470abd1e31e0bf9c3cb3 2023-02-12 14:56:22,301 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3, entries=1, sequenceid=23, filesize=5.0 K 2023-02-12 14:56:22,302 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for a9971a4f82394dd4822a56ba58fc7ed2 in 927ms, sequenceid=23, compaction requested=true 2023-02-12 14:56:22,302 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:22,302 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:22,302 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/3 local region flush tasks. 2023-02-12 14:56:22,304 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d82cb284ee204549bf195c2fe9a7a8d0 2023-02-12 14:56:22,305 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/ca8297204beb412588538111ca915513 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/ca8297204beb412588538111ca915513 2023-02-12 14:56:22,316 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc651374f4724cc79b5fe8fd723be01e 2023-02-12 14:56:22,316 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/dc651374f4724cc79b5fe8fd723be01e, entries=3, sequenceid=23, filesize=5.2 K 2023-02-12 14:56:22,317 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/bd7acbda081945a3a2659acf0135d37c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/bd7acbda081945a3a2659acf0135d37c 2023-02-12 14:56:22,321 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for bd7acbda081945a3a2659acf0135d37c 2023-02-12 14:56:22,321 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/bd7acbda081945a3a2659acf0135d37c, entries=1, sequenceid=23, filesize=5.0 K 2023-02-12 14:56:22,322 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 18a72c39300757827cdf49fe0a82a4c9 in 946ms, sequenceid=23, compaction requested=true 2023-02-12 14:56:22,322 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] regionserver.HRegion(2445): Flush status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:22,322 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-1] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:22,322 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 2/3 local region flush tasks. 2023-02-12 14:56:22,326 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca8297204beb412588538111ca915513 2023-02-12 14:56:22,326 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/ca8297204beb412588538111ca915513, entries=3, sequenceid=23, filesize=5.2 K 2023-02-12 14:56:22,327 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/d82cb284ee204549bf195c2fe9a7a8d0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d82cb284ee204549bf195c2fe9a7a8d0 2023-02-12 14:56:22,330 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d82cb284ee204549bf195c2fe9a7a8d0 2023-02-12 14:56:22,330 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d82cb284ee204549bf195c2fe9a7a8d0, entries=1, sequenceid=23, filesize=5.0 K 2023-02-12 14:56:22,331 INFO [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2947): Finished flush of dataSize ~138 B/138, heapSize ~912 B/912, currentSize=0 B/0 for 9131b0948a7ca3b4830f067f6f120964 in 954ms, sequenceid=23, compaction requested=true 2023-02-12 14:56:22,331 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] regionserver.HRegion(2445): Flush status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:22,331 DEBUG [rs(jenkins-hbase4.apache.org,45319,1676213617001)-flush-proc-pool-2] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:22,331 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 3/3 local region flush tasks. 2023-02-12 14:56:22,331 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 3 local region flush tasks. 2023-02-12 14:56:22,331 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,331 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testMajorCompactionFromAdmin' locally acquired 2023-02-12 14:56:22,331 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure (testMajorCompactionFromAdmin) in zk 2023-02-12 14:56:22,336 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:22,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,336 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testMajorCompactionFromAdmin' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-12 14:56:22,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:22,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:22,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:22,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:22,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,45319,1676213617001' joining acquired barrier for procedure 'testMajorCompactionFromAdmin' on coordinator 2023-02-12 14:56:22,338 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@852a3a1[Count = 0] remaining members to acquire global barrier 2023-02-12 14:56:22,338 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testMajorCompactionFromAdmin' starting 'in-barrier' execution. 2023-02-12 14:56:22,338 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,339 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,339 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,339 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testMajorCompactionFromAdmin' received 'reached' from coordinator. 2023-02-12 14:56:22,339 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testMajorCompactionFromAdmin' locally completed 2023-02-12 14:56:22,339 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testMajorCompactionFromAdmin' completed for member 'jenkins-hbase4.apache.org,45319,1676213617001' in zk 2023-02-12 14:56:22,339 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,339 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-12 14:56:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testMajorCompactionFromAdmin' has notified controller of completion 2023-02-12 14:56:22,340 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:22,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:22,340 DEBUG [member: 'jenkins-hbase4.apache.org,45319,1676213617001' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testMajorCompactionFromAdmin' completed. 2023-02-12 14:56:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:22,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:22,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:22,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:22,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testMajorCompactionFromAdmin' member 'jenkins-hbase4.apache.org,45319,1676213617001': 2023-02-12 14:56:22,343 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,45319,1676213617001' released barrier for procedure'testMajorCompactionFromAdmin', counting down latch. Waiting for 0 more 2023-02-12 14:56:22,343 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testMajorCompactionFromAdmin' execution completed 2023-02-12 14:56:22,343 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-12 14:56:22,343 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-12 14:56:22,343 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testMajorCompactionFromAdmin 2023-02-12 14:56:22,343 INFO [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testMajorCompactionFromAdminincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-12 14:56:22,345 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-12 14:56:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-12 14:56:22,345 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:22,345 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:22,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-12 14:56:22,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:22,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:22,345 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-12 14:56:22,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:22,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,347 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-12 14:56:22,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testMajorCompactionFromAdmin 2023-02-12 14:56:22,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,349 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:22,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-12 14:56:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-12 14:56:22,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testMajorCompactionFromAdmin' 2023-02-12 14:56:22,350 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-12 14:56:22,350 DEBUG [(jenkins-hbase4.apache.org,38665,1676213615861)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-12 14:56:22,350 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testMajorCompactionFromAdmin 2023-02-12 14:56:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testMajorCompactionFromAdmin 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testMajorCompactionFromAdmin'' to complete. (max 300000 ms per retry) 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-12 14:56:22,350 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testMajorCompactionFromAdmin 2023-02-12 14:56:22,450 DEBUG [Listener at localhost/37945] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-12 14:56:22,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=38665] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-12 14:56:22,452 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:56:22,452 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): a9971a4f82394dd4822a56ba58fc7ed2/family1 is initiating major compaction (all files) 2023-02-12 14:56:22,452 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:56:22,452 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:56:22,452 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of a9971a4f82394dd4822a56ba58fc7ed2/family1 in testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:22,452 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/cd1faa0085a041ccbbc77d5cdca99342, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/9879983a73e9452f9920f1d46d7373a6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/a7b15909520e469cb851c75f827b705e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/c68a5cf40fb84f33ade2b7d9644b9eb3] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp, totalSize=27.1 K 2023-02-12 14:56:22,453 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 4fbc66fe2b1f4f2c8bc7188aa32acf2d, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213775287 2023-02-12 14:56:22,453 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting cd1faa0085a041ccbbc77d5cdca99342, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213775990 2023-02-12 14:56:22,454 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 9879983a73e9452f9920f1d46d7373a6, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213777868 2023-02-12 14:56:22,454 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting a7b15909520e469cb851c75f827b705e, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213779764 2023-02-12 14:56:22,454 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting c68a5cf40fb84f33ade2b7d9644b9eb3, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:56:22,464 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): a9971a4f82394dd4822a56ba58fc7ed2#family1#compaction#197 average throughput is 0.57 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:22,908 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302129698b68d7c834831881422418487da7f_del 2023-02-12 14:56:22,908 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302129698b68d7c834831881422418487da7f_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302129698b68d7c834831881422418487da7f_del 2023-02-12 14:56:23,763 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c2f8e420d09463985ec063dc9fe41e8 2023-02-12 14:56:23,764 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family1/8c2f8e420d09463985ec063dc9fe41e8 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/8c2f8e420d09463985ec063dc9fe41e8 2023-02-12 14:56:23,785 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c2f8e420d09463985ec063dc9fe41e8 2023-02-12 14:56:23,786 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a9971a4f82394dd4822a56ba58fc7ed2/family1 of a9971a4f82394dd4822a56ba58fc7ed2 into 8c2f8e420d09463985ec063dc9fe41e8(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:56:23,786 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:23,786 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:56:23,786 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): a9971a4f82394dd4822a56ba58fc7ed2/family2 is initiating major compaction (all files) 2023-02-12 14:56:23,786 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:56:23,786 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:56:23,786 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of a9971a4f82394dd4822a56ba58fc7ed2/family2 in testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:23,786 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/5ea3891f1ac448f689a162ed60ceb544, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/9061beb742e246b79bda542b0d93c427, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/b5089a432a154c5abbbf3a87d96ad3dd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/f70b71de0e9a4dd7bb2a08c77991acd6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp, totalSize=25.4 K 2023-02-12 14:56:23,787 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 5ea3891f1ac448f689a162ed60ceb544, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213775287 2023-02-12 14:56:23,787 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 9061beb742e246b79bda542b0d93c427, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213775990 2023-02-12 14:56:23,788 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting b5089a432a154c5abbbf3a87d96ad3dd, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213777868 2023-02-12 14:56:23,788 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f70b71de0e9a4dd7bb2a08c77991acd6, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213779764 2023-02-12 14:56:23,788 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 2b5bb1ccce28470abd1e31e0bf9c3cb3, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:56:23,795 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): a9971a4f82394dd4822a56ba58fc7ed2#family2#compaction#198 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:23,871 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/cd1faa0085a041ccbbc77d5cdca99342, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/9879983a73e9452f9920f1d46d7373a6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/a7b15909520e469cb851c75f827b705e, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/c68a5cf40fb84f33ade2b7d9644b9eb3] to archive 2023-02-12 14:56:23,872 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:23,874 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/4fbc66fe2b1f4f2c8bc7188aa32acf2d 2023-02-12 14:56:23,875 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/cd1faa0085a041ccbbc77d5cdca99342 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/cd1faa0085a041ccbbc77d5cdca99342 2023-02-12 14:56:23,877 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/9879983a73e9452f9920f1d46d7373a6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/9879983a73e9452f9920f1d46d7373a6 2023-02-12 14:56:23,878 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/a7b15909520e469cb851c75f827b705e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/a7b15909520e469cb851c75f827b705e 2023-02-12 14:56:23,879 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/c68a5cf40fb84f33ade2b7d9644b9eb3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/c68a5cf40fb84f33ade2b7d9644b9eb3 2023-02-12 14:56:24,203 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302120791895bc85a4845b94dc2b5af601d27_del 2023-02-12 14:56:24,203 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b202302120791895bc85a4845b94dc2b5af601d27_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c4ca4238a0b923820dcc509a6f75849b202302120791895bc85a4845b94dc2b5af601d27_del 2023-02-12 14:56:25,015 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cc1282f2b124d4db54011dc4247e571 2023-02-12 14:56:25,016 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/.tmp/family2/9cc1282f2b124d4db54011dc4247e571 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/9cc1282f2b124d4db54011dc4247e571 2023-02-12 14:56:25,019 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9cc1282f2b124d4db54011dc4247e571 2023-02-12 14:56:25,020 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in a9971a4f82394dd4822a56ba58fc7ed2/family2 of a9971a4f82394dd4822a56ba58fc7ed2 into 9cc1282f2b124d4db54011dc4247e571(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:56:25,021 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:25,021 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:56:25,021 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 18a72c39300757827cdf49fe0a82a4c9/family1 is initiating major compaction (all files) 2023-02-12 14:56:25,021 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:56:25,021 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:56:25,023 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 18a72c39300757827cdf49fe0a82a4c9/family1 in testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:25,023 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f2b958201294424a8699505910a16c95, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/68b0b14982484823ab624417dcfb8d87, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f7a4044b61b84c7881d467a0722adc03, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/b4e794cc9fb14511b3a00757b7c21512, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/dc651374f4724cc79b5fe8fd723be01e] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp, totalSize=27.1 K 2023-02-12 14:56:25,024 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f2b958201294424a8699505910a16c95, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213775287 2023-02-12 14:56:25,024 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 68b0b14982484823ab624417dcfb8d87, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213775991 2023-02-12 14:56:25,024 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f7a4044b61b84c7881d467a0722adc03, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213777868 2023-02-12 14:56:25,025 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting b4e794cc9fb14511b3a00757b7c21512, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213779764 2023-02-12 14:56:25,025 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting dc651374f4724cc79b5fe8fd723be01e, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:56:25,033 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 18a72c39300757827cdf49fe0a82a4c9#family1#compaction#199 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:25,073 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/5ea3891f1ac448f689a162ed60ceb544, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/9061beb742e246b79bda542b0d93c427, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/b5089a432a154c5abbbf3a87d96ad3dd, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/f70b71de0e9a4dd7bb2a08c77991acd6, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3] to archive 2023-02-12 14:56:25,074 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:25,075 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/5ea3891f1ac448f689a162ed60ceb544 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/5ea3891f1ac448f689a162ed60ceb544 2023-02-12 14:56:25,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/9061beb742e246b79bda542b0d93c427 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/9061beb742e246b79bda542b0d93c427 2023-02-12 14:56:25,077 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/b5089a432a154c5abbbf3a87d96ad3dd to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/b5089a432a154c5abbbf3a87d96ad3dd 2023-02-12 14:56:25,078 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/f70b71de0e9a4dd7bb2a08c77991acd6 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/f70b71de0e9a4dd7bb2a08c77991acd6 2023-02-12 14:56:25,079 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family2/2b5bb1ccce28470abd1e31e0bf9c3cb3 2023-02-12 14:56:25,473 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212d882386b3c3b40c59f8fef9add8683f5_del 2023-02-12 14:56:25,473 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c20230212d882386b3c3b40c59f8fef9add8683f5_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212d882386b3c3b40c59f8fef9add8683f5_del 2023-02-12 14:56:26,331 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aaa6fc3bbeca4f90a6e4077aef776349 2023-02-12 14:56:26,331 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family1/aaa6fc3bbeca4f90a6e4077aef776349 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/aaa6fc3bbeca4f90a6e4077aef776349 2023-02-12 14:56:26,353 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aaa6fc3bbeca4f90a6e4077aef776349 2023-02-12 14:56:26,354 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 18a72c39300757827cdf49fe0a82a4c9/family1 of 18a72c39300757827cdf49fe0a82a4c9 into aaa6fc3bbeca4f90a6e4077aef776349(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:56:26,354 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:26,354 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:56:26,354 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 18a72c39300757827cdf49fe0a82a4c9/family2 is initiating major compaction (all files) 2023-02-12 14:56:26,354 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:56:26,354 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:56:26,354 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 18a72c39300757827cdf49fe0a82a4c9/family2 in testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:26,354 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/3e41637e3be042cd8a51f56a875b522f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/2e85281cd4c44099a6bdbf47df6e7ff5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/71a4039d96ec481abffc241f81792ae8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/13575bfebf3740f38c3f28b303a7f823, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/bd7acbda081945a3a2659acf0135d37c] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp, totalSize=25.4 K 2023-02-12 14:56:26,355 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 3e41637e3be042cd8a51f56a875b522f, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213775287 2023-02-12 14:56:26,355 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 2e85281cd4c44099a6bdbf47df6e7ff5, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213775991 2023-02-12 14:56:26,355 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 71a4039d96ec481abffc241f81792ae8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213777868 2023-02-12 14:56:26,356 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 13575bfebf3740f38c3f28b303a7f823, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213779764 2023-02-12 14:56:26,356 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting bd7acbda081945a3a2659acf0135d37c, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:56:26,363 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 18a72c39300757827cdf49fe0a82a4c9#family2#compaction#200 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:26,369 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f2b958201294424a8699505910a16c95, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/68b0b14982484823ab624417dcfb8d87, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f7a4044b61b84c7881d467a0722adc03, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/b4e794cc9fb14511b3a00757b7c21512, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/dc651374f4724cc79b5fe8fd723be01e] to archive 2023-02-12 14:56:26,370 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:26,371 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f2b958201294424a8699505910a16c95 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f2b958201294424a8699505910a16c95 2023-02-12 14:56:26,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/68b0b14982484823ab624417dcfb8d87 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/68b0b14982484823ab624417dcfb8d87 2023-02-12 14:56:26,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f7a4044b61b84c7881d467a0722adc03 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/f7a4044b61b84c7881d467a0722adc03 2023-02-12 14:56:26,374 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/b4e794cc9fb14511b3a00757b7c21512 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/b4e794cc9fb14511b3a00757b7c21512 2023-02-12 14:56:26,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/dc651374f4724cc79b5fe8fd723be01e to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/dc651374f4724cc79b5fe8fd723be01e 2023-02-12 14:56:26,772 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c202302127b5958159ef7471294f9637c0fe109d0_del 2023-02-12 14:56:26,772 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c202302127b5958159ef7471294f9637c0fe109d0_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/c81e728d9d4c2f636f067f89cc14862c202302127b5958159ef7471294f9637c0fe109d0_del 2023-02-12 14:56:27,586 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31e51811315249cbb8e973186d52acad 2023-02-12 14:56:27,587 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/.tmp/family2/31e51811315249cbb8e973186d52acad as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/31e51811315249cbb8e973186d52acad 2023-02-12 14:56:27,590 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31e51811315249cbb8e973186d52acad 2023-02-12 14:56:27,591 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 18a72c39300757827cdf49fe0a82a4c9/family2 of 18a72c39300757827cdf49fe0a82a4c9 into 31e51811315249cbb8e973186d52acad(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:56:27,591 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:27,591 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:56:27,591 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 9131b0948a7ca3b4830f067f6f120964/family1 is initiating major compaction (all files) 2023-02-12 14:56:27,591 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:56:27,591 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:56:27,592 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 9131b0948a7ca3b4830f067f6f120964/family1 in testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:27,592 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a0a03715c3384576be5b4cc537dd8e0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a6a22c390ac9428aba57a12d959bd2ab, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/f67e2f196f284a2690f96eb444217615, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/4f47ecb8410243bbb347e6e8c4d75e7f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/ca8297204beb412588538111ca915513] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp, totalSize=27.1 K 2023-02-12 14:56:27,592 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting a0a03715c3384576be5b4cc537dd8e0c, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213775286 2023-02-12 14:56:27,592 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting a6a22c390ac9428aba57a12d959bd2ab, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213775990 2023-02-12 14:56:27,593 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting f67e2f196f284a2690f96eb444217615, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213777868 2023-02-12 14:56:27,593 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 4f47ecb8410243bbb347e6e8c4d75e7f, keycount=4, bloomtype=ROW, size=5.5 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213779763 2023-02-12 14:56:27,593 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting ca8297204beb412588538111ca915513, keycount=3, bloomtype=ROW, size=5.2 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:56:27,601 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 9131b0948a7ca3b4830f067f6f120964#family1#compaction#201 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:27,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/3e41637e3be042cd8a51f56a875b522f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/2e85281cd4c44099a6bdbf47df6e7ff5, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/71a4039d96ec481abffc241f81792ae8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/13575bfebf3740f38c3f28b303a7f823, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/bd7acbda081945a3a2659acf0135d37c] to archive 2023-02-12 14:56:27,670 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:27,672 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/3e41637e3be042cd8a51f56a875b522f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/3e41637e3be042cd8a51f56a875b522f 2023-02-12 14:56:27,673 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/2e85281cd4c44099a6bdbf47df6e7ff5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/2e85281cd4c44099a6bdbf47df6e7ff5 2023-02-12 14:56:27,674 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/71a4039d96ec481abffc241f81792ae8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/71a4039d96ec481abffc241f81792ae8 2023-02-12 14:56:27,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/13575bfebf3740f38c3f28b303a7f823 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/13575bfebf3740f38c3f28b303a7f823 2023-02-12 14:56:27,675 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/bd7acbda081945a3a2659acf0135d37c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family2/bd7acbda081945a3a2659acf0135d37c 2023-02-12 14:56:28,037 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230212e35dff6f30174aa79ff2d44e25b44497_del 2023-02-12 14:56:28,037 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e20230212e35dff6f30174aa79ff2d44e25b44497_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230212e35dff6f30174aa79ff2d44e25b44497_del 2023-02-12 14:56:28,890 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c128f5e0bce4ec0bf4f97efa8c43aba 2023-02-12 14:56:28,891 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family1/6c128f5e0bce4ec0bf4f97efa8c43aba as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/6c128f5e0bce4ec0bf4f97efa8c43aba 2023-02-12 14:56:28,912 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6c128f5e0bce4ec0bf4f97efa8c43aba 2023-02-12 14:56:28,913 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9131b0948a7ca3b4830f067f6f120964/family1 of 9131b0948a7ca3b4830f067f6f120964 into 6c128f5e0bce4ec0bf4f97efa8c43aba(size=7.0 K), total size for store is 7.0 K. This selection was in queue for 0sec, and took 1sec to execute. 2023-02-12 14:56:28,913 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:28,913 DEBUG [Listener at localhost/37945] compactions.SortedCompactionPolicy(75): Selecting compaction from 5 store files, 0 compacting, 5 eligible, 16 blocking 2023-02-12 14:56:28,913 DEBUG [Listener at localhost/37945] regionserver.HStore(1912): 9131b0948a7ca3b4830f067f6f120964/family2 is initiating major compaction (all files) 2023-02-12 14:56:28,913 INFO [Listener at localhost/37945] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-12 14:56:28,914 INFO [Listener at localhost/37945] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-12 14:56:28,914 INFO [Listener at localhost/37945] regionserver.HRegion(2258): Starting compaction of 9131b0948a7ca3b4830f067f6f120964/family2 in testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:28,914 INFO [Listener at localhost/37945] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/6274e7dd267642a8a7320a22d406ff27, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d9aa061f9cdd495c833bc17098dfe3a8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/befd5edcbe694722805cf968247c12e9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/b247a3be051e47938971f6b8e6f611d7, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d82cb284ee204549bf195c2fe9a7a8d0] into tmpdir=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp, totalSize=25.4 K 2023-02-12 14:56:28,914 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting 6274e7dd267642a8a7320a22d406ff27, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=5, earliestPutTs=1676213775286 2023-02-12 14:56:28,914 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting d9aa061f9cdd495c833bc17098dfe3a8, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=9, earliestPutTs=1676213775990 2023-02-12 14:56:28,915 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting befd5edcbe694722805cf968247c12e9, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=13, earliestPutTs=1676213777868 2023-02-12 14:56:28,915 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting b247a3be051e47938971f6b8e6f611d7, keycount=2, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=17, earliestPutTs=1676213779763 2023-02-12 14:56:28,915 DEBUG [Listener at localhost/37945] compactions.Compactor(207): Compacting d82cb284ee204549bf195c2fe9a7a8d0, keycount=1, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=23, earliestPutTs=9223372036854775807 2023-02-12 14:56:28,922 INFO [Listener at localhost/37945] throttle.PressureAwareThroughputController(145): 9131b0948a7ca3b4830f067f6f120964#family2#compaction#202 average throughput is 0.53 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-12 14:56:28,927 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e202302125c59e5e251474f19b1789a151e7186b2_del 2023-02-12 14:56:28,928 INFO [Listener at localhost/37945] regionserver.HMobStore(257): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e202302125c59e5e251474f19b1789a151e7186b2_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family2/d41d8cd98f00b204e9800998ecf8427e202302125c59e5e251474f19b1789a151e7186b2_del 2023-02-12 14:56:28,972 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a0a03715c3384576be5b4cc537dd8e0c, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a6a22c390ac9428aba57a12d959bd2ab, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/f67e2f196f284a2690f96eb444217615, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/4f47ecb8410243bbb347e6e8c4d75e7f, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/ca8297204beb412588538111ca915513] to archive 2023-02-12 14:56:28,973 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:28,974 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a0a03715c3384576be5b4cc537dd8e0c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a0a03715c3384576be5b4cc537dd8e0c 2023-02-12 14:56:28,975 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a6a22c390ac9428aba57a12d959bd2ab to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/a6a22c390ac9428aba57a12d959bd2ab 2023-02-12 14:56:28,976 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/f67e2f196f284a2690f96eb444217615 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/f67e2f196f284a2690f96eb444217615 2023-02-12 14:56:28,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/4f47ecb8410243bbb347e6e8c4d75e7f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/4f47ecb8410243bbb347e6e8c4d75e7f 2023-02-12 14:56:28,977 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/ca8297204beb412588538111ca915513 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/ca8297204beb412588538111ca915513 2023-02-12 14:56:29,338 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27e5e12a7a104bb3b9e356ac239a24e6 2023-02-12 14:56:29,338 DEBUG [Listener at localhost/37945] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/.tmp/family2/27e5e12a7a104bb3b9e356ac239a24e6 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/27e5e12a7a104bb3b9e356ac239a24e6 2023-02-12 14:56:29,342 INFO [Listener at localhost/37945] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 27e5e12a7a104bb3b9e356ac239a24e6 2023-02-12 14:56:29,343 INFO [Listener at localhost/37945] regionserver.HStore(1652): Completed major compaction of 5 (all) file(s) in 9131b0948a7ca3b4830f067f6f120964/family2 of 9131b0948a7ca3b4830f067f6f120964 into 27e5e12a7a104bb3b9e356ac239a24e6(size=6.2 K), total size for store is 6.2 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-12 14:56:29,343 DEBUG [Listener at localhost/37945] regionserver.HRegion(2288): Compaction status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:29,346 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:56:29,349 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:56:29,352 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:56:29,353 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:56:29,358 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=38665] master.MasterMobCompactionThread(90): The mob compaction is requested for the columns [{NAME => 'family1', ENCRYPTION => 'AES', MOB_THRESHOLD => '0', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', ENCRYPTION_KEY => 'B\x0A\x03AES\x10\x10\x1A\x10\xDB\xBAP&\xA8S\x83\xCE$3'XV\x1EL\x82"\x10\xD3\x13`\xCE\xC6p\x86N\xB8\xC4T\xBB\xC4\xAB0\xC2*\x10\x89\xA8\xAD\x80\x0C;\xBA:\xCAp\xEF5\x7F\x91\x8F\xA62\x03MD5', REPLICATION_SCOPE => '0', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', IS_MOB => 'true', COMPRESSION => 'NONE', BLOCKCACHE => 'true', BLOCKSIZE => '65536'}] of the table testMajorCompactionFromAdmin 2023-02-12 14:56:29,369 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] regionserver.HStore(2712): Moving the files [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/6274e7dd267642a8a7320a22d406ff27, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d9aa061f9cdd495c833bc17098dfe3a8, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/befd5edcbe694722805cf968247c12e9, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/b247a3be051e47938971f6b8e6f611d7, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d82cb284ee204549bf195c2fe9a7a8d0] to archive 2023-02-12 14:56:29,370 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:29,372 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/6274e7dd267642a8a7320a22d406ff27 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/6274e7dd267642a8a7320a22d406ff27 2023-02-12 14:56:29,373 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d9aa061f9cdd495c833bc17098dfe3a8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d9aa061f9cdd495c833bc17098dfe3a8 2023-02-12 14:56:29,375 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/befd5edcbe694722805cf968247c12e9 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/befd5edcbe694722805cf968247c12e9 2023-02-12 14:56:29,376 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/b247a3be051e47938971f6b8e6f611d7 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/b247a3be051e47938971f6b8e6f611d7 2023-02-12 14:56:29,377 DEBUG [RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d82cb284ee204549bf195c2fe9a7a8d0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family2/d82cb284ee204549bf195c2fe9a7a8d0 2023-02-12 14:56:29,383 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] procedure2.ProcedureExecutor(1029): Stored pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-12 14:56:29,384 DEBUG [PEWorker-2] locking.LockProcedure(309): LOCKED pid=123, state=RUNNABLE; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-12 14:56:29,385 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(833): start MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:56:29,385 INFO [PEWorker-2] procedure2.TimeoutExecutorThread(81): ADDED pid=123, state=WAITING_TIMEOUT, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE; timeout=600000, timestamp=1676214389385 2023-02-12 14:56:29,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(142): is allFiles: true 2023-02-12 14:56:29,740 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(306): The compaction type is ALL_FILES, the request has 3 del files, 12 selected files, and 0 irrelevant files table 'testMajorCompactionFromAdmin' and column 'family1' 2023-02-12 14:56:29,760 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for d41d8cd98f00b204e9800998ecf8427e20230212e35dff6f30174aa79ff2d44e25b44497_del 2023-02-12 14:56:29,781 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c4ca4238a0b923820dcc509a6f75849b202302129698b68d7c834831881422418487da7f_del 2023-02-12 14:56:29,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c81e728d9d4c2f636f067f89cc14862c20230212d882386b3c3b40c59f8fef9add8683f5_del 2023-02-12 14:56:29,801 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(350): After merging, there are 3 del files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:29,802 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] zookeeper.ReadOnlyZKClient(139): Connect 0x49de2adc to 127.0.0.1:61565 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-12 14:56:29,806 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@469bb216, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-12 14:56:29,807 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c4ca4238a0b923820dcc509a6f75849b20230212 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-12 14:56:30,539 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 14:56:30,765 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c4ca4238a0b923820dcc509a6f75849b2023021289ff6c5ed80842b3b5bfe2351eca16d1 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021289ff6c5ed80842b3b5bfe2351eca16d1 2023-02-12 14:56:30,766 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:30,777 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-12 14:56:30,779 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:34170, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-12 14:56:30,787 DEBUG [MobCompactor-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-12 14:56:30,789 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:59326, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-12 14:56:30,827 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230212/family1/595fdadb1fc9411ba9fc49cacb489557 first=Optional[1\x00\x00\x00\x01] last=Optional[1\x00\x00\x00\x07] 2023-02-12 14:56:30,828 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row 1 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230212/family1/595fdadb1fc9411ba9fc49cacb489557}] 2023-02-12 14:56:30,832 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230212/family1/595fdadb1fc9411ba9fc49cacb489557 for inclusion in a9971a4f82394dd4822a56ba58fc7ed2/family1 2023-02-12 14:56:30,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=1\x00\x00\x00\x01 last=1\x00\x00\x00\x07 2023-02-12 14:56:30,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=1 last=2 2023-02-12 14:56:30,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for a9971a4f82394dd4822a56ba58fc7ed2: 2023-02-12 14:56:30,853 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230212/family1/595fdadb1fc9411ba9fc49cacb489557 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__2rum3vog2ep467i4uaklcss70hgq5p33h6vaullu7f8n0b2oo549pcm8ji64s7mv/family1/595fdadb1fc9411ba9fc49cacb489557 2023-02-12 14:56:30,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__2rum3vog2ep467i4uaklcss70hgq5p33h6vaullu7f8n0b2oo549pcm8ji64s7mv/family1/595fdadb1fc9411ba9fc49cacb489557 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/e97b426b19cd49dca9714285fbb5a253_SeqId_28_ 2023-02-12 14:56:30,855 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__2rum3vog2ep467i4uaklcss70hgq5p33h6vaullu7f8n0b2oo549pcm8ji64s7mv/family1/595fdadb1fc9411ba9fc49cacb489557 into a9971a4f82394dd4822a56ba58fc7ed2/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/e97b426b19cd49dca9714285fbb5a253_SeqId_28_ - updating store file list. 2023-02-12 14:56:30,878 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/e97b426b19cd49dca9714285fbb5a253_SeqId_28_ into a9971a4f82394dd4822a56ba58fc7ed2/family1 2023-02-12 14:56:30,879 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__2rum3vog2ep467i4uaklcss70hgq5p33h6vaullu7f8n0b2oo549pcm8ji64s7mv/family1/595fdadb1fc9411ba9fc49cacb489557 into a9971a4f82394dd4822a56ba58fc7ed2/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/family1/e97b426b19cd49dca9714285fbb5a253_SeqId_28_) 2023-02-12 14:56:30,879 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__2rum3vog2ep467i4uaklcss70hgq5p33h6vaullu7f8n0b2oo549pcm8ji64s7mv/family1/595fdadb1fc9411ba9fc49cacb489557 2023-02-12 14:56:30,881 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c4ca4238a0b923820dcc509a6f75849b20230212 table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:30,882 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:30,883 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302120298c9e7962346c9afa3322176122b1f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302120298c9e7962346c9afa3322176122b1f 2023-02-12 14:56:30,884 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302123e7c8237cbd74151908117ad92ecb428 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302123e7c8237cbd74151908117ad92ecb428 2023-02-12 14:56:30,885 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021273486115f203438db23a69471c7aaec5 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b2023021273486115f203438db23a69471c7aaec5 2023-02-12 14:56:30,886 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230212c2b45f15790643528b12d40ef6b65ce0 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b20230212c2b45f15790643528b12d40ef6b65ce0 2023-02-12 14:56:30,886 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-12 14:56:30,887 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c4ca4238a0b923820dcc509a6f75849b20230212' 2023-02-12 14:56:30,887 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c4ca4238a0b923820dcc509a6f75849b20230212 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-12 14:56:30,887 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition c81e728d9d4c2f636f067f89cc14862c20230212 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-12 14:56:31,780 DEBUG [regionserver/jenkins-hbase4:0.Chore.2] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 14:56:31,837 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/c81e728d9d4c2f636f067f89cc14862c2023021219b2c1d607ea4184b3f0b33fe08ee36b to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021219b2c1d607ea4184b3f0b33fe08ee36b 2023-02-12 14:56:31,838 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:31,880 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230212/family1/9fcd317ab89e4de98903227f26b8a39c first=Optional[2\x00\x00\x00\x01] last=Optional[2\x00\x00\x00\x07] 2023-02-12 14:56:31,881 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row 2 with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230212/family1/9fcd317ab89e4de98903227f26b8a39c}] 2023-02-12 14:56:31,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230212/family1/9fcd317ab89e4de98903227f26b8a39c for inclusion in 18a72c39300757827cdf49fe0a82a4c9/family1 2023-02-12 14:56:31,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=2\x00\x00\x00\x01 last=2\x00\x00\x00\x07 2023-02-12 14:56:31,904 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first=2 last= 2023-02-12 14:56:31,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 18a72c39300757827cdf49fe0a82a4c9: 2023-02-12 14:56:31,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230212/family1/9fcd317ab89e4de98903227f26b8a39c to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__7m8b8a22jfe99h0tuo11tddfuen5pt820qpfbahk0f6om6n8t9amfropme0q3lbc/family1/9fcd317ab89e4de98903227f26b8a39c 2023-02-12 14:56:31,907 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__7m8b8a22jfe99h0tuo11tddfuen5pt820qpfbahk0f6om6n8t9amfropme0q3lbc/family1/9fcd317ab89e4de98903227f26b8a39c as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/a59d1785d445476f99ff088283fadf40_SeqId_28_ 2023-02-12 14:56:31,907 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__7m8b8a22jfe99h0tuo11tddfuen5pt820qpfbahk0f6om6n8t9amfropme0q3lbc/family1/9fcd317ab89e4de98903227f26b8a39c into 18a72c39300757827cdf49fe0a82a4c9/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/a59d1785d445476f99ff088283fadf40_SeqId_28_ - updating store file list. 2023-02-12 14:56:31,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/a59d1785d445476f99ff088283fadf40_SeqId_28_ into 18a72c39300757827cdf49fe0a82a4c9/family1 2023-02-12 14:56:31,929 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__7m8b8a22jfe99h0tuo11tddfuen5pt820qpfbahk0f6om6n8t9amfropme0q3lbc/family1/9fcd317ab89e4de98903227f26b8a39c into 18a72c39300757827cdf49fe0a82a4c9/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/family1/a59d1785d445476f99ff088283fadf40_SeqId_28_) 2023-02-12 14:56:31,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__7m8b8a22jfe99h0tuo11tddfuen5pt820qpfbahk0f6om6n8t9amfropme0q3lbc/family1/9fcd317ab89e4de98903227f26b8a39c 2023-02-12 14:56:31,931 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition c81e728d9d4c2f636f067f89cc14862c20230212 table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:31,931 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:31,932 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021231a56592fc364779b7a62e0d168f3a8d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021231a56592fc364779b7a62e0d168f3a8d 2023-02-12 14:56:31,933 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021290c59c318a8d4acebc27f718c933b0c8 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c2023021290c59c318a8d4acebc27f718c933b0c8 2023-02-12 14:56:31,934 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212d0fbfd30751d49b8bc25f35012e85474 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212d0fbfd30751d49b8bc25f35012e85474 2023-02-12 14:56:31,935 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212ea4c31f842854a6c842c19e50fcf5b1d to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212ea4c31f842854a6c842c19e50fcf5b1d 2023-02-12 14:56:31,935 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-12 14:56:31,935 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/c81e728d9d4c2f636f067f89cc14862c20230212' 2023-02-12 14:56:31,936 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=c81e728d9d4c2f636f067f89cc14862c20230212 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-12 14:56:31,936 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor$1(495): Compacting mob files for partition d41d8cd98f00b204e9800998ecf8427e20230212 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-12 14:56:32,503 INFO [MobCompactor-pool-0] mob.MobUtils(725): Renaming flushed file from hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/d41d8cd98f00b204e9800998ecf8427e2023021270c0b01a785748a28a86a9195d017b1f to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021270c0b01a785748a28a86a9195d017b1f 2023-02-12 14:56:32,504 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(688): start MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:32,553 INFO [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles(787): Trying to load hfile=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230212/family1/28e3578bef1e401dba7b27c469de54a7 first=Optional[0\x00\x00\x00\x01] last=Optional[0\x00\x00\x00\x07] 2023-02-12 14:56:32,554 DEBUG [LoadIncrementalHFiles-0] tool.LoadIncrementalHFiles$1(552): Going to connect to server region=testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964., hostname=jenkins-hbase4.apache.org,45319,1676213617001, seqNum=2 for row with hfile group [{family1,hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230212/family1/28e3578bef1e401dba7b27c469de54a7}] 2023-02-12 14:56:32,556 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(763): Validating hfile at hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230212/family1/28e3578bef1e401dba7b27c469de54a7 for inclusion in 9131b0948a7ca3b4830f067f6f120964/family1 2023-02-12 14:56:32,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(775): HFile bounds: first=0\x00\x00\x00\x01 last=0\x00\x00\x00\x07 2023-02-12 14:56:32,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(777): Region bounds: first= last=1 2023-02-12 14:56:32,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegion(2445): Flush status journal for 9131b0948a7ca3b4830f067f6f120964: 2023-02-12 14:56:32,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(386): Moving hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230212/family1/28e3578bef1e401dba7b27c469de54a7 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__shbf9hu6knu0v53jc9a12vbtnjiv0smp9chpr76c8bkl37ohfq530nip8g781qsq/family1/28e3578bef1e401dba7b27c469de54a7 2023-02-12 14:56:32,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__shbf9hu6knu0v53jc9a12vbtnjiv0smp9chpr76c8bkl37ohfq530nip8g781qsq/family1/28e3578bef1e401dba7b27c469de54a7 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/c8d70bd701ec4729b668eb7882a35315_SeqId_28_ 2023-02-12 14:56:32,580 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(849): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__shbf9hu6knu0v53jc9a12vbtnjiv0smp9chpr76c8bkl37ohfq530nip8g781qsq/family1/28e3578bef1e401dba7b27c469de54a7 into 9131b0948a7ca3b4830f067f6f120964/family1 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/c8d70bd701ec4729b668eb7882a35315_SeqId_28_ - updating store file list. 2023-02-12 14:56:32,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(882): Loaded HFile hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/c8d70bd701ec4729b668eb7882a35315_SeqId_28_ into 9131b0948a7ca3b4830f067f6f120964/family1 2023-02-12 14:56:32,602 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.HStore(855): Successfully loaded hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__shbf9hu6knu0v53jc9a12vbtnjiv0smp9chpr76c8bkl37ohfq530nip8g781qsq/family1/28e3578bef1e401dba7b27c469de54a7 into 9131b0948a7ca3b4830f067f6f120964/family1 (new location: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/family1/c8d70bd701ec4729b668eb7882a35315_SeqId_28_) 2023-02-12 14:56:32,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=45319] regionserver.SecureBulkLoadManager$SecureBulkLoadListener(399): Bulk Load done for: hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/staging/jenkins__testMajorCompactionFromAdmin__shbf9hu6knu0v53jc9a12vbtnjiv0smp9chpr76c8bkl37ohfq530nip8g781qsq/family1/28e3578bef1e401dba7b27c469de54a7 2023-02-12 14:56:32,604 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(692): end MOB ref bulkload for partition d41d8cd98f00b204e9800998ecf8427e20230212 table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:32,605 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:32,606 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302123cc6f37097454d4d8917120d52a07ecf to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302123cc6f37097454d4d8917120d52a07ecf 2023-02-12 14:56:32,607 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302124d552cbb411d4b4a8b6c96b6d8553aba to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302124d552cbb411d4b4a8b6c96b6d8553aba 2023-02-12 14:56:32,608 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021276a93aad046b4596a8fe53da99c7961a to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e2023021276a93aad046b4596a8fe53da99c7961a 2023-02-12 14:56:32,609 DEBUG [MobCompactor-pool-0] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302129785bb7af6884ab1a573690f51770179 to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e202302129785bb7af6884ab1a573690f51770179 2023-02-12 14:56:32,609 INFO [MobCompactor-pool-0] mob.MobUtils(508): Table testMajorCompactionFromAdmin 4 expired mob files are deleted 2023-02-12 14:56:32,609 DEBUG [MobCompactor-pool-0] compactions.PartitionedMobCompactor(917): Cleanup, delete path 'hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/.tmp/.bulkload/default/testMajorCompactionFromAdmin/d41d8cd98f00b204e9800998ecf8427e20230212' 2023-02-12 14:56:32,610 INFO [MobCompactor-pool-0] compactions.PartitionedMobCompactor(582): Compaction is finished. The number of mob files is changed from 4 to 1 for partition=d41d8cd98f00b204e9800998ecf8427e20230212 for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-12 14:56:32,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(354): After compaction, there are 3 mob files. table='testMajorCompactionFromAdmin' column='family1' 2023-02-12 14:56:32,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(364): After a mob compaction with all files selected, archiving the del files for table='testMajorCompactionFromAdmin' and column='family1' 2023-02-12 14:56:32,610 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230212e35dff6f30174aa79ff2d44e25b44497_del] 2023-02-12 14:56:32,610 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:32,611 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230212e35dff6f30174aa79ff2d44e25b44497_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/d41d8cd98f00b204e9800998ecf8427e20230212e35dff6f30174aa79ff2d44e25b44497_del 2023-02-12 14:56:32,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-12 14:56:32,611 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302129698b68d7c834831881422418487da7f_del] 2023-02-12 14:56:32,612 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:32,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302129698b68d7c834831881422418487da7f_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c4ca4238a0b923820dcc509a6f75849b202302129698b68d7c834831881422418487da7f_del 2023-02-12 14:56:32,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-12 14:56:32,613 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] compactions.PartitionedMobCompactor(367): [hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212d882386b3c3b40c59f8fef9add8683f5_del] 2023-02-12 14:56:32,613 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] backup.HFileArchiver(360): Archiving compacted files. 2023-02-12 14:56:32,615 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/mobdir/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212d882386b3c3b40c59f8fef9add8683f5_del to hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/archive/data/default/testMajorCompactionFromAdmin/85b2cf7da420f757b7327e61350f7918/family1/c81e728d9d4c2f636f067f89cc14862c20230212d882386b3c3b40c59f8fef9add8683f5_del 2023-02-12 14:56:32,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(508): Table testMajorCompactionFromAdmin 1 expired mob files are deleted 2023-02-12 14:56:32,615 INFO [master/jenkins-hbase4:0:becomeActiveMaster-MasterMobCompaction-1676213620509] mob.MobUtils(840): end MOB compaction of files for table='testMajorCompactionFromAdmin', column='family1', allFiles=true, compactor='class org.apache.hadoop.hbase.mob.compactions.PartitionedMobCompactor' 2023-02-12 14:56:32,616 DEBUG [PEWorker-1] locking.LockProcedure(242): UNLOCKED pid=123, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE 2023-02-12 14:56:32,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, state=SUCCESS; org.apache.hadoop.hbase.master.locking.LockProcedure, tableName=testMajorCompactionFromAdmin.mobLock, type=EXCLUSIVE in 3.2330 sec 2023-02-12 14:56:32,630 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:56:32,632 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:56:32,634 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '1', inclusive 2023-02-12 14:56:32,635 DEBUG [Listener at localhost/37945] client.ClientScanner(240): Advancing internal scanner to startKey at '2', inclusive 2023-02-12 14:56:32,639 INFO [Listener at localhost/37945] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-12 14:56:32,677 INFO [Listener at localhost/37945] hbase.ResourceChecker(175): after: mob.compactions.TestMobCompactor#testMajorCompactionFromAdmin Thread=247 (was 253), OpenFileDescriptor=644 (was 644), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=171 (was 176), ProcessCount=168 (was 168), AvailableMemoryMB=3963 (was 3978) 2023-02-12 14:56:32,678 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5831e476 to 127.0.0.1:61565 2023-02-12 14:56:32,678 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:32,678 INFO [Listener at localhost/37945] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-12 14:56:32,678 INFO [Listener at localhost/37945] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-12 14:56:32,679 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x13c53277 to 127.0.0.1:61565 2023-02-12 14:56:32,679 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:32,679 DEBUG [Listener at localhost/37945] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-12 14:56:32,679 DEBUG [Listener at localhost/37945] util.JVMClusterUtil(257): Found active master hash=1156374039, stopped=false 2023-02-12 14:56:32,679 INFO [Listener at localhost/37945] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:56:32,681 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-12 14:56:32,681 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-12 14:56:32,681 INFO [Listener at localhost/37945] procedure2.ProcedureExecutor(629): Stopping 2023-02-12 14:56:32,681 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-12 14:56:32,682 DEBUG [Listener at localhost/37945] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0b3d6d3c to 127.0.0.1:61565 2023-02-12 14:56:32,682 DEBUG [Listener at localhost/37945] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:32,682 INFO [Listener at localhost/37945] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,45319,1676213617001' ***** 2023-02-12 14:56:32,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 14:56:32,682 INFO [Listener at localhost/37945] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-12 14:56:32,682 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-12 14:56:32,682 INFO [RS:0;jenkins-hbase4:45319] regionserver.HeapMemoryManager(220): Stopping 2023-02-12 14:56:32,683 INFO [RS:0;jenkins-hbase4:45319] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-12 14:56:32,683 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-12 14:56:32,683 INFO [RS:0;jenkins-hbase4:45319] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-12 14:56:32,683 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 27e64129bc866c604e8923b6312886ba 2023-02-12 14:56:32,683 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for a9971a4f82394dd4822a56ba58fc7ed2 2023-02-12 14:56:32,683 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 27e64129bc866c604e8923b6312886ba, disabling compactions & flushes 2023-02-12 14:56:32,683 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for aac84b1ca5b7fcfec0ef433f2e9b7077 2023-02-12 14:56:32,684 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 618f7278d7d2801ef6c08ce41da650ea 2023-02-12 14:56:32,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 7c8a55283e262bc196e8394e111484b6 2023-02-12 14:56:32,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. after waiting 0 ms 2023-02-12 14:56:32,684 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for ac5d58614983383110f00b3bb838c006 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 73ab03d290bf433ec02678876bd209ea 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for a9e7a8a10a47fb644927b2cbb23ae591 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 18a72c39300757827cdf49fe0a82a4c9 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 0384a0af3ce8daed2b5dc2a6da84edf7 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 9131b0948a7ca3b4830f067f6f120964 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for b4d026fc325cd5a11dc0dcb67fe52664 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 1af670086541a56fea5102c106da6986 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 6a7186591bf78b4981ff4a06040b98fe 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 05e3c4702d173182c255943db0b5df6e 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 98f05e4653d1b15554e6b39d5c185fff 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 8aa6e64a63f09f579f8ce0eebfb86e36 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for c330f294c6804231199503c93a0f53d8 2023-02-12 14:56:32,684 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 966e620c05fc39d1026d2c1ff17e6679 2023-02-12 14:56:32,685 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:32,685 DEBUG [RS:0;jenkins-hbase4:45319] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x46c2a243 to 127.0.0.1:61565 2023-02-12 14:56:32,685 DEBUG [RS:0;jenkins-hbase4:45319] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:32,685 INFO [RS:0;jenkins-hbase4:45319] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-12 14:56:32,685 INFO [RS:0;jenkins-hbase4:45319] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-12 14:56:32,685 INFO [RS:0;jenkins-hbase4:45319] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-12 14:56:32,685 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-12 14:56:32,686 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1475): Waiting on 21 regions to close 2023-02-12 14:56:32,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-12 14:56:32,686 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-12 14:56:32,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-12 14:56:32,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-12 14:56:32,686 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-12 14:56:32,686 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=50.56 KB heapSize=78.23 KB 2023-02-12 14:56:32,687 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1505): Waiting on 0384a0af3ce8daed2b5dc2a6da84edf7, 05e3c4702d173182c255943db0b5df6e, 1588230740, 18a72c39300757827cdf49fe0a82a4c9, 1af670086541a56fea5102c106da6986, 27e64129bc866c604e8923b6312886ba, 618f7278d7d2801ef6c08ce41da650ea, 6a7186591bf78b4981ff4a06040b98fe, 73ab03d290bf433ec02678876bd209ea, 7c8a55283e262bc196e8394e111484b6, 8aa6e64a63f09f579f8ce0eebfb86e36, 9131b0948a7ca3b4830f067f6f120964, 966e620c05fc39d1026d2c1ff17e6679, 98f05e4653d1b15554e6b39d5c185fff, a9971a4f82394dd4822a56ba58fc7ed2, a9e7a8a10a47fb644927b2cbb23ae591, aac84b1ca5b7fcfec0ef433f2e9b7077, ac5d58614983383110f00b3bb838c006, b4d026fc325cd5a11dc0dcb67fe52664, c330f294c6804231199503c93a0f53d8, c92bf76a80608abe270eff23b9822d0c 2023-02-12 14:56:32,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/27e64129bc866c604e8923b6312886ba/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-12 14:56:32,688 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:56:32,688 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 27e64129bc866c604e8923b6312886ba: Waiting for close lock at 1676213792683Running coprocessor pre-close hooks at 1676213792683Disabling compacts and flushes for region at 1676213792683Disabling writes for close at 1676213792684 (+1 ms)Writing region close event to WAL at 1676213792684Running coprocessor post-close hooks at 1676213792688 (+4 ms)Closed at 1676213792688 2023-02-12 14:56:32,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,,1676213659168.27e64129bc866c604e8923b6312886ba. 2023-02-12 14:56:32,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a9971a4f82394dd4822a56ba58fc7ed2, disabling compactions & flushes 2023-02-12 14:56:32,689 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:32,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:32,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. after waiting 0 ms 2023-02-12 14:56:32,689 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:32,699 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/a9971a4f82394dd4822a56ba58fc7ed2/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-12 14:56:32,700 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:32,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a9971a4f82394dd4822a56ba58fc7ed2: Waiting for close lock at 1676213792689Running coprocessor pre-close hooks at 1676213792689Disabling compacts and flushes for region at 1676213792689Disabling writes for close at 1676213792689Writing region close event to WAL at 1676213792696 (+7 ms)Running coprocessor post-close hooks at 1676213792699 (+3 ms)Closed at 1676213792700 (+1 ms) 2023-02-12 14:56:32,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,1,1676213773143.a9971a4f82394dd4822a56ba58fc7ed2. 2023-02-12 14:56:32,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing aac84b1ca5b7fcfec0ef433f2e9b7077, disabling compactions & flushes 2023-02-12 14:56:32,700 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:56:32,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:56:32,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. after waiting 0 ms 2023-02-12 14:56:32,700 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:56:32,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly/aac84b1ca5b7fcfec0ef433f2e9b7077/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-12 14:56:32,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:56:32,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for aac84b1ca5b7fcfec0ef433f2e9b7077: Waiting for close lock at 1676213792700Running coprocessor pre-close hooks at 1676213792700Disabling compacts and flushes for region at 1676213792700Disabling writes for close at 1676213792700Writing region close event to WAL at 1676213792701 (+1 ms)Running coprocessor post-close hooks at 1676213792703 (+2 ms)Closed at 1676213792704 (+1 ms) 2023-02-12 14:56:32,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByWeekly,,1676213675297.aac84b1ca5b7fcfec0ef433f2e9b7077. 2023-02-12 14:56:32,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 618f7278d7d2801ef6c08ce41da650ea, disabling compactions & flushes 2023-02-12 14:56:32,704 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:56:32,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:56:32,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. after waiting 0 ms 2023-02-12 14:56:32,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:56:32,707 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithMonthlyPolicy/618f7278d7d2801ef6c08ce41da650ea/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-12 14:56:32,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:56:32,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 618f7278d7d2801ef6c08ce41da650ea: Waiting for close lock at 1676213792704Running coprocessor pre-close hooks at 1676213792704Disabling compacts and flushes for region at 1676213792704Disabling writes for close at 1676213792704Writing region close event to WAL at 1676213792705 (+1 ms)Running coprocessor post-close hooks at 1676213792707 (+2 ms)Closed at 1676213792708 (+1 ms) 2023-02-12 14:56:32,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithMonthlyPolicy,,1676213635194.618f7278d7d2801ef6c08ce41da650ea. 2023-02-12 14:56:32,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 7c8a55283e262bc196e8394e111484b6, disabling compactions & flushes 2023-02-12 14:56:32,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:56:32,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:56:32,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. after waiting 0 ms 2023-02-12 14:56:32,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:56:32,711 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/7c8a55283e262bc196e8394e111484b6/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-12 14:56:32,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:56:32,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 7c8a55283e262bc196e8394e111484b6: Waiting for close lock at 1676213792708Running coprocessor pre-close hooks at 1676213792708Disabling compacts and flushes for region at 1676213792708Disabling writes for close at 1676213792708Writing region close event to WAL at 1676213792709 (+1 ms)Running coprocessor post-close hooks at 1676213792712 (+3 ms)Closed at 1676213792712 2023-02-12 14:56:32,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,1,1676213714066.7c8a55283e262bc196e8394e111484b6. 2023-02-12 14:56:32,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing ac5d58614983383110f00b3bb838c006, disabling compactions & flushes 2023-02-12 14:56:32,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:56:32,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:56:32,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. after waiting 0 ms 2023-02-12 14:56:32,712 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:56:32,715 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyPolicy/ac5d58614983383110f00b3bb838c006/recovered.edits/49.seqid, newMaxSeqId=49, maxSeqId=1 2023-02-12 14:56:32,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:56:32,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for ac5d58614983383110f00b3bb838c006: Waiting for close lock at 1676213792712Running coprocessor pre-close hooks at 1676213792712Disabling compacts and flushes for region at 1676213792712Disabling writes for close at 1676213792712Writing region close event to WAL at 1676213792713 (+1 ms)Running coprocessor post-close hooks at 1676213792715 (+2 ms)Closed at 1676213792716 (+1 ms) 2023-02-12 14:56:32,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyPolicy,,1676213663174.ac5d58614983383110f00b3bb838c006. 2023-02-12 14:56:32,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 73ab03d290bf433ec02678876bd209ea, disabling compactions & flushes 2023-02-12 14:56:32,716 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:56:32,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:56:32,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. after waiting 0 ms 2023-02-12 14:56:32,716 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:56:32,720 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/73ab03d290bf433ec02678876bd209ea/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-12 14:56:32,721 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:56:32,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 73ab03d290bf433ec02678876bd209ea: Waiting for close lock at 1676213792716Running coprocessor pre-close hooks at 1676213792716Disabling compacts and flushes for region at 1676213792716Disabling writes for close at 1676213792716Writing region close event to WAL at 1676213792718 (+2 ms)Running coprocessor post-close hooks at 1676213792720 (+2 ms)Closed at 1676213792721 (+1 ms) 2023-02-12 14:56:32,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,2,1676213714066.73ab03d290bf433ec02678876bd209ea. 2023-02-12 14:56:32,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing a9e7a8a10a47fb644927b2cbb23ae591, disabling compactions & flushes 2023-02-12 14:56:32,721 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-12 14:56:32,721 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:56:32,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:56:32,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. after waiting 0 ms 2023-02-12 14:56:32,721 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:56:32,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/a9e7a8a10a47fb644927b2cbb23ae591/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-12 14:56:32,724 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:56:32,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for a9e7a8a10a47fb644927b2cbb23ae591: Waiting for close lock at 1676213792721Running coprocessor pre-close hooks at 1676213792721Disabling compacts and flushes for region at 1676213792721Disabling writes for close at 1676213792721Writing region close event to WAL at 1676213792722 (+1 ms)Running coprocessor post-close hooks at 1676213792724 (+2 ms)Closed at 1676213792724 2023-02-12 14:56:32,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,2,1676213742623.a9e7a8a10a47fb644927b2cbb23ae591. 2023-02-12 14:56:32,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 18a72c39300757827cdf49fe0a82a4c9, disabling compactions & flushes 2023-02-12 14:56:32,725 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:32,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:32,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. after waiting 0 ms 2023-02-12 14:56:32,725 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:32,728 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/18a72c39300757827cdf49fe0a82a4c9/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-12 14:56:32,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:32,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 18a72c39300757827cdf49fe0a82a4c9: Waiting for close lock at 1676213792725Running coprocessor pre-close hooks at 1676213792725Disabling compacts and flushes for region at 1676213792725Disabling writes for close at 1676213792725Writing region close event to WAL at 1676213792726 (+1 ms)Running coprocessor post-close hooks at 1676213792729 (+3 ms)Closed at 1676213792729 2023-02-12 14:56:32,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,2,1676213773143.18a72c39300757827cdf49fe0a82a4c9. 2023-02-12 14:56:32,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 0384a0af3ce8daed2b5dc2a6da84edf7, disabling compactions & flushes 2023-02-12 14:56:32,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:56:32,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:56:32,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. after waiting 0 ms 2023-02-12 14:56:32,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:56:32,733 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthly/0384a0af3ce8daed2b5dc2a6da84edf7/recovered.edits/55.seqid, newMaxSeqId=55, maxSeqId=50 2023-02-12 14:56:32,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:56:32,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 0384a0af3ce8daed2b5dc2a6da84edf7: Waiting for close lock at 1676213792730Running coprocessor pre-close hooks at 1676213792730Disabling compacts and flushes for region at 1676213792730Disabling writes for close at 1676213792730Writing region close event to WAL at 1676213792731 (+1 ms)Running coprocessor post-close hooks at 1676213792733 (+2 ms)Closed at 1676213792734 (+1 ms) 2023-02-12 14:56:32,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthly,,1676213620632.0384a0af3ce8daed2b5dc2a6da84edf7. 2023-02-12 14:56:32,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 9131b0948a7ca3b4830f067f6f120964, disabling compactions & flushes 2023-02-12 14:56:32,734 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:32,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:32,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. after waiting 0 ms 2023-02-12 14:56:32,734 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:32,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionFromAdmin/9131b0948a7ca3b4830f067f6f120964/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-12 14:56:32,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:32,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 9131b0948a7ca3b4830f067f6f120964: Waiting for close lock at 1676213792734Running coprocessor pre-close hooks at 1676213792734Disabling compacts and flushes for region at 1676213792734Disabling writes for close at 1676213792734Writing region close event to WAL at 1676213792735 (+1 ms)Running coprocessor post-close hooks at 1676213792737 (+2 ms)Closed at 1676213792738 (+1 ms) 2023-02-12 14:56:32,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionFromAdmin,,1676213773143.9131b0948a7ca3b4830f067f6f120964. 2023-02-12 14:56:32,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing b4d026fc325cd5a11dc0dcb67fe52664, disabling compactions & flushes 2023-02-12 14:56:32,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:56:32,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:56:32,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. after waiting 0 ms 2023-02-12 14:56:32,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:56:32,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2744): Flushing b4d026fc325cd5a11dc0dcb67fe52664 1/1 column families, dataSize=109 B heapSize=592 B 2023-02-12 14:56:32,744 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=109 B at sequenceid=7 (bloomFilter=true), to=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/.tmp/info/c45fc5dc6fee43ab9935fde962c9d0c4 2023-02-12 14:56:32,749 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/.tmp/info/c45fc5dc6fee43ab9935fde962c9d0c4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/info/c45fc5dc6fee43ab9935fde962c9d0c4 2023-02-12 14:56:32,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/info/c45fc5dc6fee43ab9935fde962c9d0c4, entries=3, sequenceid=7, filesize=4.8 K 2023-02-12 14:56:32,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for b4d026fc325cd5a11dc0dcb67fe52664 in 15ms, sequenceid=7, compaction requested=false 2023-02-12 14:56:32,757 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/namespace/b4d026fc325cd5a11dc0dcb67fe52664/recovered.edits/10.seqid, newMaxSeqId=10, maxSeqId=1 2023-02-12 14:56:32,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:56:32,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for b4d026fc325cd5a11dc0dcb67fe52664: Waiting for close lock at 1676213792738Running coprocessor pre-close hooks at 1676213792738Disabling compacts and flushes for region at 1676213792738Disabling writes for close at 1676213792738Obtaining lock to block concurrent updates at 1676213792738Preparing flush snapshotting stores in b4d026fc325cd5a11dc0dcb67fe52664 at 1676213792738Finished memstore snapshotting hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664., syncing WAL and waiting on mvcc, flushsize=dataSize=109, getHeapSize=576, getOffHeapSize=0, getCellsCount=3 at 1676213792738Flushing stores of hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. at 1676213792739 (+1 ms)Flushing b4d026fc325cd5a11dc0dcb67fe52664/info: creating writer at 1676213792739Flushing b4d026fc325cd5a11dc0dcb67fe52664/info: appending metadata at 1676213792741 (+2 ms)Flushing b4d026fc325cd5a11dc0dcb67fe52664/info: closing flushed file at 1676213792741Flushing b4d026fc325cd5a11dc0dcb67fe52664/info: reopening flushed file at 1676213792749 (+8 ms)Finished flush of dataSize ~109 B/109, heapSize ~576 B/576, currentSize=0 B/0 for b4d026fc325cd5a11dc0dcb67fe52664 in 15ms, sequenceid=7, compaction requested=false at 1676213792754 (+5 ms)Writing region close event to WAL at 1676213792755 (+1 ms)Running coprocessor post-close hooks at 1676213792757 (+2 ms)Closed at 1676213792758 (+1 ms) 2023-02-12 14:56:32,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676213619487.b4d026fc325cd5a11dc0dcb67fe52664. 2023-02-12 14:56:32,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 1af670086541a56fea5102c106da6986, disabling compactions & flushes 2023-02-12 14:56:32,758 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:56:32,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:56:32,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. after waiting 0 ms 2023-02-12 14:56:32,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:56:32,762 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithWeeklyPolicy/1af670086541a56fea5102c106da6986/recovered.edits/50.seqid, newMaxSeqId=50, maxSeqId=1 2023-02-12 14:56:32,763 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:56:32,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1af670086541a56fea5102c106da6986: Waiting for close lock at 1676213792758Running coprocessor pre-close hooks at 1676213792758Disabling compacts and flushes for region at 1676213792758Disabling writes for close at 1676213792758Writing region close event to WAL at 1676213792760 (+2 ms)Running coprocessor post-close hooks at 1676213792762 (+2 ms)Closed at 1676213792763 (+1 ms) 2023-02-12 14:56:32,763 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithWeeklyPolicy,,1676213646721.1af670086541a56fea5102c106da6986. 2023-02-12 14:56:32,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 6a7186591bf78b4981ff4a06040b98fe, disabling compactions & flushes 2023-02-12 14:56:32,764 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:56:32,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:56:32,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. after waiting 0 ms 2023-02-12 14:56:32,764 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:56:32,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/6a7186591bf78b4981ff4a06040b98fe/recovered.edits/15.seqid, newMaxSeqId=15, maxSeqId=1 2023-02-12 14:56:32,766 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:56:32,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 6a7186591bf78b4981ff4a06040b98fe: Waiting for close lock at 1676213792763Running coprocessor pre-close hooks at 1676213792764 (+1 ms)Disabling compacts and flushes for region at 1676213792764Disabling writes for close at 1676213792764Writing region close event to WAL at 1676213792764Running coprocessor post-close hooks at 1676213792766 (+2 ms)Closed at 1676213792766 2023-02-12 14:56:32,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,2,1676213659168.6a7186591bf78b4981ff4a06040b98fe. 2023-02-12 14:56:32,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 05e3c4702d173182c255943db0b5df6e, disabling compactions & flushes 2023-02-12 14:56:32,766 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:56:32,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:56:32,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. after waiting 0 ms 2023-02-12 14:56:32,767 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:56:32,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/05e3c4702d173182c255943db0b5df6e/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-12 14:56:32,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:56:32,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 05e3c4702d173182c255943db0b5df6e: Waiting for close lock at 1676213792766Running coprocessor pre-close hooks at 1676213792766Disabling compacts and flushes for region at 1676213792766Disabling writes for close at 1676213792767 (+1 ms)Writing region close event to WAL at 1676213792768 (+1 ms)Running coprocessor post-close hooks at 1676213792770 (+2 ms)Closed at 1676213792770 2023-02-12 14:56:32,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,1,1676213742623.05e3c4702d173182c255943db0b5df6e. 2023-02-12 14:56:32,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 98f05e4653d1b15554e6b39d5c185fff, disabling compactions & flushes 2023-02-12 14:56:32,770 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:32,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:32,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. after waiting 0 ms 2023-02-12 14:56:32,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:32,776 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactedHFilesCleaner was stopped 2023-02-12 14:56:32,778 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMinorCompactionWithMonthlyPolicy/98f05e4653d1b15554e6b39d5c185fff/recovered.edits/52.seqid, newMaxSeqId=52, maxSeqId=1 2023-02-12 14:56:32,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:32,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 98f05e4653d1b15554e6b39d5c185fff: Waiting for close lock at 1676213792770Running coprocessor pre-close hooks at 1676213792770Disabling compacts and flushes for region at 1676213792770Disabling writes for close at 1676213792771 (+1 ms)Writing region close event to WAL at 1676213792776 (+5 ms)Running coprocessor post-close hooks at 1676213792779 (+3 ms)Closed at 1676213792779 2023-02-12 14:56:32,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMinorCompactionWithMonthlyPolicy,,1676213759451.98f05e4653d1b15554e6b39d5c185fff. 2023-02-12 14:56:32,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 8aa6e64a63f09f579f8ce0eebfb86e36, disabling compactions & flushes 2023-02-12 14:56:32,779 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:56:32,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:56:32,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. after waiting 0 ms 2023-02-12 14:56:32,779 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:56:32,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily/8aa6e64a63f09f579f8ce0eebfb86e36/recovered.edits/58.seqid, newMaxSeqId=58, maxSeqId=55 2023-02-12 14:56:32,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:56:32,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 8aa6e64a63f09f579f8ce0eebfb86e36: Waiting for close lock at 1676213792779Running coprocessor pre-close hooks at 1676213792779Disabling compacts and flushes for region at 1676213792779Disabling writes for close at 1676213792779Writing region close event to WAL at 1676213792780 (+1 ms)Running coprocessor post-close hooks at 1676213792782 (+2 ms)Closed at 1676213792782 2023-02-12 14:56:32,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testMajorCompactionWithWeeklyFollowedByMonthlyFollowedByDaily,,1676213692348.8aa6e64a63f09f579f8ce0eebfb86e36. 2023-02-12 14:56:32,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c92bf76a80608abe270eff23b9822d0c, disabling compactions & flushes 2023-02-12 14:56:32,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:56:32,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:56:32,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. after waiting 0 ms 2023-02-12 14:56:32,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:56:32,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/ns/testMinorCompaction/c92bf76a80608abe270eff23b9822d0c/recovered.edits/30.seqid, newMaxSeqId=30, maxSeqId=1 2023-02-12 14:56:32,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:56:32,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c92bf76a80608abe270eff23b9822d0c: Waiting for close lock at 1676213792782Running coprocessor pre-close hooks at 1676213792782Disabling compacts and flushes for region at 1676213792782Disabling writes for close at 1676213792782Writing region close event to WAL at 1676213792783 (+1 ms)Running coprocessor post-close hooks at 1676213792785 (+2 ms)Closed at 1676213792785 2023-02-12 14:56:32,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed ns:testMinorCompaction,,1676213742623.c92bf76a80608abe270eff23b9822d0c. 2023-02-12 14:56:32,785 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing c330f294c6804231199503c93a0f53d8, disabling compactions & flushes 2023-02-12 14:56:32,786 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:56:32,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:56:32,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. after waiting 0 ms 2023-02-12 14:56:32,786 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:56:32,788 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testCompactionWithHFileLink/c330f294c6804231199503c93a0f53d8/recovered.edits/38.seqid, newMaxSeqId=38, maxSeqId=31 2023-02-12 14:56:32,788 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:56:32,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for c330f294c6804231199503c93a0f53d8: Waiting for close lock at 1676213792785Running coprocessor pre-close hooks at 1676213792785Disabling compacts and flushes for region at 1676213792785Disabling writes for close at 1676213792786 (+1 ms)Writing region close event to WAL at 1676213792786Running coprocessor post-close hooks at 1676213792788 (+2 ms)Closed at 1676213792788 2023-02-12 14:56:32,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testCompactionWithHFileLink,,1676213714066.c330f294c6804231199503c93a0f53d8. 2023-02-12 14:56:32,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1603): Closing 966e620c05fc39d1026d2c1ff17e6679, disabling compactions & flushes 2023-02-12 14:56:32,789 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1625): Closing region testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:56:32,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:56:32,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1713): Acquired close lock on testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. after waiting 0 ms 2023-02-12 14:56:32,789 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1723): Updates disabled for region testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:56:32,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/default/testScannerOnBulkLoadRefHFiles/966e620c05fc39d1026d2c1ff17e6679/recovered.edits/6.seqid, newMaxSeqId=6, maxSeqId=1 2023-02-12 14:56:32,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:56:32,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 966e620c05fc39d1026d2c1ff17e6679: Waiting for close lock at 1676213792789Running coprocessor pre-close hooks at 1676213792789Disabling compacts and flushes for region at 1676213792789Disabling writes for close at 1676213792789Writing region close event to WAL at 1676213792789Running coprocessor post-close hooks at 1676213792791 (+2 ms)Closed at 1676213792791 2023-02-12 14:56:32,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed testScannerOnBulkLoadRefHFiles,1,1676213659168.966e620c05fc39d1026d2c1ff17e6679. 2023-02-12 14:56:32,887 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-12 14:56:33,016 DEBUG [regionserver/jenkins-hbase4:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-12 14:56:33,087 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-12 14:56:33,100 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=48.28 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/.tmp/info/7fb5506a579b423ea145135997c5b7d9 2023-02-12 14:56:33,103 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7fb5506a579b423ea145135997c5b7d9 2023-02-12 14:56:33,156 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-02-12 14:56:33,156 INFO [regionserver/jenkins-hbase4:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-02-12 14:56:33,288 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-12 14:56:33,488 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-12 14:56:33,510 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=102 B at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/.tmp/rep_barrier/74cba85dc14c48ad81a43f7d5e04cce4 2023-02-12 14:56:33,514 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 74cba85dc14c48ad81a43f7d5e04cce4 2023-02-12 14:56:33,688 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-12 14:56:33,688 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-12 14:56:33,688 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-12 14:56:33,888 DEBUG [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-12 14:56:33,921 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.18 KB at sequenceid=130 (bloomFilter=false), to=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/.tmp/table/6fd06e57e7c44edf880d2220ef4b06d4 2023-02-12 14:56:33,925 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fd06e57e7c44edf880d2220ef4b06d4 2023-02-12 14:56:33,925 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/.tmp/info/7fb5506a579b423ea145135997c5b7d9 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/info/7fb5506a579b423ea145135997c5b7d9 2023-02-12 14:56:33,929 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 7fb5506a579b423ea145135997c5b7d9 2023-02-12 14:56:33,929 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/info/7fb5506a579b423ea145135997c5b7d9, entries=226, sequenceid=130, filesize=37.9 K 2023-02-12 14:56:33,929 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/.tmp/rep_barrier/74cba85dc14c48ad81a43f7d5e04cce4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/rep_barrier/74cba85dc14c48ad81a43f7d5e04cce4 2023-02-12 14:56:33,932 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 74cba85dc14c48ad81a43f7d5e04cce4 2023-02-12 14:56:33,932 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/rep_barrier/74cba85dc14c48ad81a43f7d5e04cce4, entries=1, sequenceid=130, filesize=5.0 K 2023-02-12 14:56:33,933 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/.tmp/table/6fd06e57e7c44edf880d2220ef4b06d4 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/table/6fd06e57e7c44edf880d2220ef4b06d4 2023-02-12 14:56:33,936 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fd06e57e7c44edf880d2220ef4b06d4 2023-02-12 14:56:33,936 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/table/6fd06e57e7c44edf880d2220ef4b06d4, entries=27, sequenceid=130, filesize=6.6 K 2023-02-12 14:56:33,938 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1252ms, sequenceid=130, compaction requested=false 2023-02-12 14:56:33,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/data/hbase/meta/1588230740/recovered.edits/133.seqid, newMaxSeqId=133, maxSeqId=1 2023-02-12 14:56:33,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-12 14:56:33,942 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-12 14:56:33,942 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676213792686Running coprocessor pre-close hooks at 1676213792686Disabling compacts and flushes for region at 1676213792686Disabling writes for close at 1676213792686Obtaining lock to block concurrent updates at 1676213792686Preparing flush snapshotting stores in 1588230740 at 1676213792686Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=51772, getHeapSize=80064, getOffHeapSize=0, getCellsCount=365 at 1676213792686Flushing stores of hbase:meta,,1.1588230740 at 1676213792687 (+1 ms)Flushing 1588230740/info: creating writer at 1676213792687Flushing 1588230740/info: appending metadata at 1676213792691 (+4 ms)Flushing 1588230740/info: closing flushed file at 1676213792691Flushing 1588230740/rep_barrier: creating writer at 1676213793104 (+413 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676213793106 (+2 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676213793106Flushing 1588230740/table: creating writer at 1676213793515 (+409 ms)Flushing 1588230740/table: appending metadata at 1676213793517 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676213793517Flushing 1588230740/info: reopening flushed file at 1676213793926 (+409 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676213793930 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676213793933 (+3 ms)Finished flush of dataSize ~50.56 KB/51772, heapSize ~78.19 KB/80064, currentSize=0 B/0 for 1588230740 in 1252ms, sequenceid=130, compaction requested=false at 1676213793938 (+5 ms)Writing region close event to WAL at 1676213793940 (+2 ms)Running coprocessor post-close hooks at 1676213793942 (+2 ms)Closed at 1676213793942 2023-02-12 14:56:33,943 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-12 14:56:34,089 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,45319,1676213617001; all regions closed. 2023-02-12 14:56:34,096 DEBUG [RS:0;jenkins-hbase4:45319] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/oldWALs 2023-02-12 14:56:34,096 INFO [RS:0;jenkins-hbase4:45319] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45319%2C1676213617001.meta:.meta(num 1676213619248) 2023-02-12 14:56:34,100 DEBUG [RS:0;jenkins-hbase4:45319] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/oldWALs 2023-02-12 14:56:34,100 INFO [RS:0;jenkins-hbase4:45319] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C45319%2C1676213617001:(num 1676213618298) 2023-02-12 14:56:34,100 DEBUG [RS:0;jenkins-hbase4:45319] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:34,100 INFO [RS:0;jenkins-hbase4:45319] regionserver.LeaseManager(133): Closed leases 2023-02-12 14:56:34,100 INFO [RS:0;jenkins-hbase4:45319] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-12 14:56:34,100 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-12 14:56:34,101 INFO [RS:0;jenkins-hbase4:45319] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:45319 2023-02-12 14:56:34,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,45319,1676213617001 2023-02-12 14:56:34,109 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 14:56:34,109 ERROR [Listener at localhost/37945-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@52d824e3 rejected from java.util.concurrent.ThreadPoolExecutor@56e7ba60[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 14:56:34,110 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-12 14:56:34,110 ERROR [Listener at localhost/37945-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@31d0fae5 rejected from java.util.concurrent.ThreadPoolExecutor@56e7ba60[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 14:56:34,111 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,45319,1676213617001] 2023-02-12 14:56:34,111 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,45319,1676213617001; numProcessing=1 2023-02-12 14:56:34,112 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,45319,1676213617001 already deleted, retry=false 2023-02-12 14:56:34,112 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,45319,1676213617001 expired; onlineServers=0 2023-02-12 14:56:34,112 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase4.apache.org,38665,1676213615861' ***** 2023-02-12 14:56:34,113 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-12 14:56:34,113 DEBUG [M:0;jenkins-hbase4:38665] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5d1d42c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-02-12 14:56:34,113 INFO [M:0;jenkins-hbase4:38665] regionserver.HRegionServer(1145): stopping server jenkins-hbase4.apache.org,38665,1676213615861 2023-02-12 14:56:34,113 INFO [M:0;jenkins-hbase4:38665] regionserver.HRegionServer(1171): stopping server jenkins-hbase4.apache.org,38665,1676213615861; all regions closed. 2023-02-12 14:56:34,113 DEBUG [M:0;jenkins-hbase4:38665] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-12 14:56:34,113 DEBUG [M:0;jenkins-hbase4:38665] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-12 14:56:34,113 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-12 14:56:34,114 DEBUG [M:0;jenkins-hbase4:38665] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-12 14:56:34,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676213617992] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1676213617992,5,FailOnTimeoutGroup] 2023-02-12 14:56:34,114 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676213617992] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1676213617992,5,FailOnTimeoutGroup] 2023-02-12 14:56:34,114 INFO [M:0;jenkins-hbase4:38665] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-12 14:56:34,114 INFO [M:0;jenkins-hbase4:38665] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-12 14:56:34,114 INFO [M:0;jenkins-hbase4:38665] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-02-12 14:56:34,115 DEBUG [M:0;jenkins-hbase4:38665] master.HMaster(1502): Stopping service threads 2023-02-12 14:56:34,115 INFO [M:0;jenkins-hbase4:38665] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-12 14:56:34,115 ERROR [M:0;jenkins-hbase4:38665] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT 2023-02-12 14:56:34,115 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-20,5,PEWorkerGroup] 2023-02-12 14:56:34,115 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase Thread[HFileArchiver-21,5,PEWorkerGroup] Thread[HFileArchiver-22,5,PEWorkerGroup] 2023-02-12 14:56:34,115 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-12 14:56:34,116 INFO [M:0;jenkins-hbase4:38665] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-12 14:56:34,116 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-12 14:56:34,116 DEBUG [M:0;jenkins-hbase4:38665] zookeeper.ZKUtil(398): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-12 14:56:34,116 WARN [M:0;jenkins-hbase4:38665] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-12 14:56:34,117 INFO [M:0;jenkins-hbase4:38665] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-12 14:56:34,117 INFO [M:0;jenkins-hbase4:38665] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-12 14:56:34,117 DEBUG [M:0;jenkins-hbase4:38665] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-12 14:56:34,117 INFO [M:0;jenkins-hbase4:38665] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:56:34,117 DEBUG [M:0;jenkins-hbase4:38665] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:56:34,117 DEBUG [M:0;jenkins-hbase4:38665] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-12 14:56:34,117 DEBUG [M:0;jenkins-hbase4:38665] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:56:34,117 INFO [M:0;jenkins-hbase4:38665] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=512.73 KB heapSize=598.45 KB 2023-02-12 14:56:34,211 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 14:56:34,211 INFO [RS:0;jenkins-hbase4:45319] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,45319,1676213617001; zookeeper connection closed. 2023-02-12 14:56:34,211 ERROR [Listener at localhost/37945-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@49e3b59d rejected from java.util.concurrent.ThreadPoolExecutor@56e7ba60[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 14:56:34,211 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): regionserver:45319-0x1004905e3890001, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 14:56:34,211 ERROR [Listener at localhost/37945-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@457b0ad9 rejected from java.util.concurrent.ThreadPoolExecutor@56e7ba60[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 532] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 14:56:34,211 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@1ee0ba6c] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@1ee0ba6c 2023-02-12 14:56:34,212 INFO [Listener at localhost/37945] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 1 regionserver(s) complete 2023-02-12 14:56:34,526 INFO [M:0;jenkins-hbase4:38665] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=512.73 KB at sequenceid=1005 (bloomFilter=true), to=hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/786c8fc5cc604cf6ade0350191e797c0 2023-02-12 14:56:34,529 INFO [M:0;jenkins-hbase4:38665] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 786c8fc5cc604cf6ade0350191e797c0 2023-02-12 14:56:34,530 DEBUG [M:0;jenkins-hbase4:38665] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/786c8fc5cc604cf6ade0350191e797c0 as hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/786c8fc5cc604cf6ade0350191e797c0 2023-02-12 14:56:34,533 INFO [M:0;jenkins-hbase4:38665] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 786c8fc5cc604cf6ade0350191e797c0 2023-02-12 14:56:34,533 INFO [M:0;jenkins-hbase4:38665] regionserver.HStore(1080): Added hdfs://localhost:45093/user/jenkins/test-data/1a7160fd-c48c-2ee4-eb5f-08d29d194985/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/786c8fc5cc604cf6ade0350191e797c0, entries=123, sequenceid=1005, filesize=33.3 K 2023-02-12 14:56:34,534 INFO [M:0;jenkins-hbase4:38665] regionserver.HRegion(2947): Finished flush of dataSize ~512.73 KB/525032, heapSize ~598.44 KB/612800, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 417ms, sequenceid=1005, compaction requested=false 2023-02-12 14:56:34,535 INFO [M:0;jenkins-hbase4:38665] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-12 14:56:34,535 DEBUG [M:0;jenkins-hbase4:38665] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676213794117Disabling compacts and flushes for region at 1676213794117Disabling writes for close at 1676213794117Obtaining lock to block concurrent updates at 1676213794117Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676213794117Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=525032, getHeapSize=612800, getOffHeapSize=0, getCellsCount=1160 at 1676213794117Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676213794118 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676213794118Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676213794122 (+4 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676213794122Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676213794530 (+408 ms)Finished flush of dataSize ~512.73 KB/525032, heapSize ~598.44 KB/612800, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 417ms, sequenceid=1005, compaction requested=false at 1676213794534 (+4 ms)Writing region close event to WAL at 1676213794535 (+1 ms)Closed at 1676213794535 2023-02-12 14:56:34,537 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-12 14:56:34,537 INFO [M:0;jenkins-hbase4:38665] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-12 14:56:34,538 INFO [M:0;jenkins-hbase4:38665] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:38665 2023-02-12 14:56:34,540 DEBUG [M:0;jenkins-hbase4:38665] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,38665,1676213615861 already deleted, retry=false 2023-02-12 14:56:34,642 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 14:56:34,642 INFO [M:0;jenkins-hbase4:38665] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase4.apache.org,38665,1676213615861; zookeeper connection closed. 2023-02-12 14:56:34,642 ERROR [Listener at localhost/37945-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3d9edbd2 rejected from java.util.concurrent.ThreadPoolExecutor@2363b043[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 14:56:34,642 DEBUG [Listener at localhost/37945-EventThread] zookeeper.ZKWatcher(600): master:38665-0x1004905e3890000, quorum=127.0.0.1:61565, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-12 14:56:34,642 ERROR [Listener at localhost/37945-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4db332b6 rejected from java.util.concurrent.ThreadPoolExecutor@2363b043[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 906] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-12 14:56:34,644 WARN [Listener at localhost/37945] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-12 14:56:34,647 INFO [Listener at localhost/37945] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-12 14:56:34,755 WARN [BP-1695427604-172.31.14.131-1676213612779 heartbeating to localhost/127.0.0.1:45093] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-12 14:56:34,755 WARN [BP-1695427604-172.31.14.131-1676213612779 heartbeating to localhost/127.0.0.1:45093] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1695427604-172.31.14.131-1676213612779 (Datanode Uuid 2330fbb2-a4ad-4699-a334-233f2dae637e) service to localhost/127.0.0.1:45093 2023-02-12 14:56:34,756 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1/dfs/data/data1/current/BP-1695427604-172.31.14.131-1676213612779] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 14:56:34,756 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/92177e3d-f747-3529-093b-d4a72a8f63be/cluster_fbf33812-0889-c443-717e-a7455c9d03c1/dfs/data/data2/current/BP-1695427604-172.31.14.131-1676213612779] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-12 14:56:34,785 INFO [Listener at localhost/37945] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-12 14:56:34,931 INFO [Listener at localhost/37945] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-12 14:56:35,032 INFO [Listener at localhost/37945] hbase.HBaseTestingUtility(1293): Minicluster is down